https://orcid.org/0000-0001-8635-9521
Abstract
Objective:
Over the past decades European societies have become increasingly diverse. This diversity in culture, education, and language significantly impacts neuropsychological assessment. Although several initiatives are under way to overcome these barriers – e.g. newly developed and validated test batteries – there is a need for more collaboration in the development and implementation of neuropsychological tests, such as in the domains of social cognition and language.
Method:
To address these gaps in cross-cultural neuropsychological assessment in Europe, the European Consortium on Cross-Cultural Neuropsychology (ECCroN) was established in 2019.
Results:
ECCroN recommends taking a broad range of variables into account, such as linguistic factors, literacy, education, migration history, acculturation and other cultural factors. We advocate against race-based norms as a solution to the challenging interpretation of group differences on neuropsychological tests, and instead support the development, validation, and standardization of more widely applicable/cross-culturally applicable tests that take into account interindividual variability. Last, ECCroN advocates for an improvement in the clinical training of neuropsychologists in culturally sensitive neuropsychological assessment, and the development and implementation of guidelines for interpreter-mediated neuropsychological assessment in diverse populations in Europe.
Conclusions:
ECCroN may impact research and clinical practice by contributing to existing theoretical frameworks and by improving the assessment of diverse individuals across Europe through collaborations on test development, collection of normative data, cross-cultural clinical training, and interpreter-mediated assessment.
In this position paper, we provide a general overview of the challenges to and status of cross-cultural neuropsychological assessment in Europe, and subsequently present the standpoints and potential impact of the newly formed European Consortium on Cross-Cultural Neuropsychology (ECCroN). These standpoints reflect the emerging scientific evidence in cross-cultural neuropsychology in Europe as well as the combined clinical and research experience of the individual consortium members.
Europe: a continent with unique challenges to neuropsychological assessment
Over the past decades European societies have become increasingly diverse. Following decolonialization in the second half of the twentieth century, inhabitants of former European colonies immigrated to European countries, such as North Africans in France and Afro-Caribbean and South-Asian people in the United Kingdom (van Mol & de Valk, Citation2016). Furthermore, in times of economic prosperity, European countries have traditionally relied upon a (low-educated) labor force recruited in countries outside and within the European Union to carry out low-skilled labor work (Diez Guardia & Pichelmann, Citation2006; van Mol & de Valk, Citation2016). In addition, refugees and asylum seekers have fled to Europe from the 1980s onwards (Diez Guardia & Pichelmann, Citation2006; United Nations High Commissioner for Refugees, Citation2001). Combined with those individuals born in European countries, including indigenous minorities such as the Sámi in the northern parts of Norway, Sweden, and Finland, Travelers in Ireland and the United Kingdom, and Romani people throughout Europe, this makes for a strikingly culturally and educationally diverse European population. Furthermore, Europe is characterized by remarkable linguistic diversity. In addition to the languages spoken by those from outside Europe, many of Europe’s inhabitants are bilingual or multilingual, and foreign language learning is part of the school curriculum in most European countries (European Commission/EACEA/Eurydice, Citation2017). Many countries have multiple official languages and/or nationally recognized/(co-)official dialects; for example, in Spain, people may speak Spanish, as well as other languages such as Catalan, Basque, Galician (Spanish Const. art. 3, Citation1978), or several other dialects.
Several of these populations – but particularly the “guest workers” who immigrated to Europe between 1950–1974 – are at risk of developing cognitive impairment, due to a higher prevalence of medical conditions such as diabetes (Kunst et al., Citation2011), stroke (Kunst et al., Citation2011), hypertension (van Laer et al., Citation2018) and dementia disorders (Selten et al., Citation2020). Inevitably, neuropsychologists will therefore increasingly encounter culturally, linguistically, and educationally diverse individuals in clinical practice. However, this diversity significantly impacts neuropsychological assessment, and assessment practices therefore need to be adapted to suit these diverse populations – a need that has been internationally recognized (e.g. Rivera Mindt et al., Citation2010).
A recent Delphi study (Franzen et al., Citation2020) revealed that several initiatives are under way to address some of the most urgent issues in adult cross-cultural neuropsychology in Europe, such as the development of memory tests and screening tools that may support a culture-sensitive cognitive assessment. One important tool to result from these initiatives is the European Cross-Cultural Neuropsychological Test Battery (Nielsen et al., 2018). The Delphi study also highlighted a need for more collaboration in the development, publishing, and implementation of neuropsychological tests developed in Europe, as well as a need for more research in the domains of social cognition and language in particular. In addition, it revealed pressing matters regarding training clinicians in cross-cultural neuropsychological assessment and working with interpreters in interpreter-mediated assessments. However, these issues are not specific to adult cross-cultural neuropsychology, as the field of pediatric neuropsychology faces similar – as well as unique – challenges to cross-cultural neuropsychological assessment.
To address these gaps in cross-cultural neuropsychological assessment in Europe, ECCroN was established in late 2019 by 16 specialists from ten countries; founding consortium members represent the Netherlands, Denmark, Norway, England, Scotland, France, Spain, and Italy, as well as two specialists from the United States of America and the State of Palestine working on multinational projects with one or more European site(s). ECCroN is currently actively reaching out to specialists working with pediatric and adult diverse populations in other European countries and invites others working in European contexts to join the consortium by reaching out to the consortium members. ECCroN convenes in monthly to bimonthly web-based video conferences, as well as at European conferences, such as the biennial meeting of the Federation of European Neuropsychological Societies (FESN). In the next paragraphs, we outline the main standpoints of ECCroN.
Towards a broad definition and measurement of diversity: ECCroN recommends taking into account lifetime demographics and contextual factors
ECCroN proposes a broad definition of diversity in neuropsychology; instead of just studying those born in different countries or those of different ethnic groups, ECCroN recommends taking a broad range of variables into account, such as linguistic factors (e.g. dialect, age of second language acquisition), literacy, education, migration history, acculturation and other cultural factors, as well as other relevant social determinants of health (see e.g. Glymour & Manly, Citation2008). For example, most neuropsychological tests have been developed for educated people and may not be suitable for individuals with low literacy skills, regardless of their country of origin. In addition, norms that are representative of the cultural, educational, and linguistic diversity in Europe are lacking for most tests.
In the United States, many informative group level variables may be drawn from state or national databases, such as relevant indicators of educational quality – the length of the school term, the average number of school days the student attended, and the student to teacher ratio (Glymour & Manly, Citation2008). These factors provide additional value alongside traditional self-report variables, such as urban versus rural location of the school and whether single primary school lessons comprised children of several ages. As such regional or national data are often unavailable in Europe – even more so in those who immigrated from countries outside of Europe – collaborative approaches are needed to better measure and take diversity-related variables into account. A step in this direction would be to explore whether these variables are currently being measured by researchers and clinicians, and if so, how they are operationalized. This may lead to a recommended set of variables to consider in research and clinical assessment of diverse individuals across Europe – containing, for example, suitable measures of acculturation or educational quality. The ECCroN consortium members have started working towards this goal by structurally taking inventory of whether/how each of the aspects in the ECLECTIC framework (Fujii, Citation2018) are currently being measured by the consortium members. This framework encompasses Education (level, quality, literacy), Culture and acculturation, Language (spoken and proficiency in the majority language), Economic issues, Communication style, the Testing situation (including comfort and motivation), the conceptualization of Intelligence, and the Context of immigration.
ECCroN supports the use of widely applicable cross-cultural tests as opposed to race-based norms
A recent study in a small sample of European experts in cross-cultural neuropsychology indicated that appropriate norms were not available for some – and sometimes even for none – of the available tests used in the clinic, or that norms were only available for some populations (Franzen et al., Citation2020). Race-based norms, which are commonly – but controversially (Manly & Echemendia, Citation2007) – applied in countries such as the United States, are not widely used in Europe. This should be seen in light of the historic events in Europe during World War II and subsequent European policies: EU member states generally have strict legislation regarding data collection by race – and to a lesser degree ethnicity – to prevent discrimination (Farkas, Citation2017).
Given the marked diversity in Europe and the controversial nature of race-based norms, ECCroN advocates against race-based norms as a solution to the challenging interpretation of group differences on neuropsychological tests. Instead, ECCroN aims to focus on the development, validation, and standardization of more widely applicable/cross-culturally applicable tests. Many traditional neuropsychological tests, such as the Trail Making Test, are unsuitable for diverse populations, due to their reliance on school-based skills such as reading and writing (in the Latin alphabet) and the culturally-specific abstract reasoning skills they require (Fernandez & Marcopulos, Citation2008; Franzen, van den Berg, et al., 2020; Ganguli et al., Citation1996). In addition, tests emphasizing speed may be less suitable as cultural differences exist in time perception (Agranovich et al., Citation2011) – e.g. a good result may be considered as contingent upon a slow, thorough process (Agranovich & Puente, Citation2007; Ardila, Citation2005). Applying race-based norms on tests in which floor effects are likely to occur due to factors other than cognitive impairment will preclude valid conclusions on true cognitive functioning. For example, healthy Turkish immigrants who are illiterate often show floor performance on common tasks of visuoconstruction, such as Clock Drawing and figure copy (Nielsen & Jorgensen, Citation2013); applying a normative correction to such a performance would make it hard to document impaired test performance and will inadvertently lead to the misclassification of persons with cognitive impairment as cognitively normal. In addition, caution should be exercised before using tests that are proclaimed to be “culture-free” but that have never been studied in diverse populations – the use of such non-adapted “culture-free” test may lead to diagnostic mistakes or misclassifications (Daugherty et al., Citation2017; Lozano-Ruiz et al., Citation2021). For example, applying Spanish or British norms for the Raven’s Colored Progressive Matrices – a test that has historically been labeled “culture-free” due to its minimal linguistic requirements – to a sample of normally developing Moroccan children resulted in a substantial number of children being classified as having “below average” or “impaired” intelligence (Lozano-Ruiz et al., Citation2021).
ECCroN therefore supports the use of more widely applicable, cross-cultural tests. Such tests should tap into the same cognitive ability in individuals across different cultures (construct validity) and be psychometrically sound, e.g. show clear differences in performance between persons with and without cognitive impairment (no floor effects). In addition, the influence of cultural factors such as acculturation on test performance should be minimal. These tests should use widely applicable stimuli instead of culture-specific ones – e.g. avoiding items like the igloo, pretzel, and beaver used in the Boston Naming Test (Barker-Collo, Citation2001; Chen et al., Citation2014) – and should not rely on (school-based) skills that patients likely never acquired. The concepts and instructions of these widely applicable tests should be clear and easy to understand, even for those who are not used to being tested, i.e. have limited “test-wiseness” (Aghvinian et al., Citation2020; Ardila, Citation2007). Although we recognize that standardized testing is important, ECCroN recommends to actively create an environment in which diverse patients feel comfortable and will perform optimally; in some cases, this means that additional explanations are necessary to ensure patients understand the need to undergo testing and the instructions for each individual test. ECCroN also recommends that researchers follow the adaptation and translation procedures outlined by the International Test Commission (Citation2017) when applying existing tests to a population the test was not designed for. ECCroN is actively represented in the workgroup of the International Neuropsychological Society’s Cultural Special Interest Group that is working on a neuropsychological comment on the ICT guidelines.
Several of such widely-applicable instruments have already been developed over the years across Europe, such as the aforementioned European Cross-Cultural Neuropsychological Test Battery (CNTB, Nielsen et al., 2018), the Multicultural Cognitive Examination (MCE, Nielsen et al., Citation2019), the Cross-Cultural Dementia Screening (CCD, Goudsmit et al., Citation2017), the computerized EMBRACED battery (Ibanez-Casas et al., in press), the computerized Battery for Neuropsychological Evaluation of Children (BENCI, Fasfous et al., 2015), an innovative verbal fluency-switching task (TFA-93, Narme et al., Citation2019) and a number of culturally – or regionally – appropriate picture-based memory tests, such as the Recall of Pictures Test (RPT, Nielsen et al., Citation2012), modified Visual Association Test (mVAT, Franzen et al., Citation2019), TMA-93 (Maillet et al., Citation2017), and TNI-93 (Maillet et al., Citation2016). A normative data and validation study was carried out for European majority groups, Pakistani/Indian, Polish, Turkish, and to a lesser extent Moroccan and Former Yugoslavian participants for CNTB (using multilingual research assistants or trained interpreters); for the CCD, the normative data and validation study was conducted among native Dutch, Moroccan-Arabic, Moroccan-Amazigh (Berber), Turkish, Surinamese-Creole, and Surinamese-Hindustani participants (assessed by bilingual, bicultural neuropsychologists); a general multicultural immigrant population as well as native French individuals were studied for the normative data and validity studies of the TNI-93, TMA-93, and TFA-93 (assessment in French). Normative data is increasingly collected for other tests; in addition, smaller normative data sets that were not formally published are available for some tests, such as a sample of predominantly Turkish individuals for the mVAT, which was validated in multicultural memory clinics across the Netherlands.
By using these more widely applicable tests that can be administered with an interpreter present, many cultural and linguistic effects can be minimalized (e.g. Al-Jawahiri & Nielsen, Citation2020). Ultimately, such tests may prove more feasible in diverse patient populations, and reducing cultural and linguistic effects will make the interpretation of the results of the neuropsychological assessment less challenging. The influence of education and literacy on test performance seem to be the most difficult to reduce in test design. In some cases, education-based norms will therefore remain necessary, although some recent paradigms using ecologically relevant material, some with low linguistic demand, show promise in that respect as well (Watermeyer & Calia, Citation2019). ECCroN consortium members are currently developing and validating tests measuring less well-studied cognitive domains in diverse individuals, such as language and social cognition, as well as brief tools that can be used to screen for cognitive impairment in general practice.
Becoming more sensitive to diversity: ECCroN recommends improvements in clinician training and the use of interpreters
Ideally, the European workforce of neuropsychologists would reflect the level of diversity within European societies; in current reality it is likely far from that ideal. For example, current selection criteria result in an underrepresentation of Black and Asian applicants in doctoral programs in (clinical) psychology (Scior et al., Citation2007), and Black and Asian individuals are underrepresented among National Health Services psychologists in comparison to the general population (Health and Social Care Information Centre, Citation2013). Although data on diversity among neuropsychologists is lacking in other European countries, both experts in cross-cultural neuropsychology (Franzen et al., Citation2020) and the Cultural and Ethnic Diversity Taskforce of the European Federation of Psychologists’ Associations (Birk Jensen & Kolman, Citation2018) have previously recognized diversity among the professional workforce of (neuro)psychologists as an important issue. Currently, it is not clear which factors contribute to this underrepresentation of diverse (neuro)psychologists in Europe, and to what degree these issues vary across Europe; however, it is likely that these mechanisms will vary by country due to variation in factors such as entry criteria and selection procedures (e.g. no selection, selection based on grade point averages, selection based on assessment) and the accessibility of graduate and postgraduate education (e.g. tuition fees) across European countries. More research is urgently needed to shed light on the mechanisms behind the underrepresentation, before any targeted actions can be undertaken in the form of, for example, mentoring programs or changes to selection procedures.
However, even if diversity levels were to improve, it is unlikely that it will be possible to provide same-ethnicity providers to every patient; for example, major cities in the Netherlands like Rotterdam and Amsterdam represent more than 170 nationalities (van Oosteren et al., Citation2013), while nationwide, there are only 161 neuropsychologists registered under the protected title of clinical neuropsychologist – who supervise a small subset of neuropsychologists among the 14,641 nationally registered health care psychologists (CIBG, Citation2021). While recognizing the potential benefits of assessments conducted by same-ethnicity neuropsychologists, such as outlined by e.g. Byrd et al. (Citation2010), ECCroN therefore advocates for a general improvement in the clinical training of all neuropsychologists in cross-cultural neuropsychological assessment, to ensure patient-friendly communication and correct administration and interpretation of cross-cultural neuropsychological tests. ECCroN is specifically investigating the development of a best practice that includes the minimal requirements for carrying out cross-cultural neuropsychological assessment, drawing inspiration from previous work by international (neuro)psychologists, such as the “Guidelines on multicultural education, training, research, practice, and organizational change for psychologists” by the American Psychological Association (Citation2003) and the work by Fujii (Citation2018). In addition to a best practice, ECCroN is currently working towards cross-cultural clinical training at a European level, such as a European summer school or post-master course in cross-cultural neuropsychological assessment. To this end, ECCroN has started to collect and integrate existing training materials in Europe that were identified in a previous study (Franzen et al., Citation2020). We particularly endorse European-level training as a first step, as integrating cross-cultural neuropsychology training in all individual national neuropsychology curricula is challenging given the variation in the duration, level, and content of training in neuropsychology across European countries (Hokkanen et al., Citation2019). A European program ensures good accessibility, particularly for neuropsychologists working in countries in which cross-cultural neuropsychology is less developed. This program may provide state-of-the-art knowledge through physical or virtual lectures held by ECCroN members, conveying the latest evidence-based practices from the international literature. Country specific add-ons to this European summer school or post-master course can subsequently be developed if needed. After this European program has been established, ECCroN aims to contribute to the integration of cross-cultural neuropsychology in national pre- and postgraduate training programs in neuropsychology.
Last, guidelines for interpreter-mediated neuropsychological assessment in diverse populations in Europe should be developed or adapted from existing guidelines for working with interpreters in psychological/medical practice, e.g. those of the British Psychological Society (Tribe & Thompson, Citation2017). These guidelines should cover several aspects; for example, they may describe how to brief interpreters before the neuropsychological assessment about the aims of the assessment and its standardized test procedures (Franzen et al., Citation2020). It may also cover aspects such as the disadvantages of interpreter-mediated assessment via telephone, issues with regional variations in languages (e.g. Spanish in patients from South America) and issues with interpreters who are not certified (Franzen et al., Citation2020; Judd et al., Citation2009).
The potential impact of ECCroN
ECCroN may impact research and clinical practice in several ways. First, it may accelerate improvements in assessment and subsequent diagnosis of diverse individuals in Europe. Such improvements are urgently needed; for example, previous European work has indicated that dementia is likely over-diagnosed in diverse individuals younger than 60 years and underdiagnosed in those older than 60 (Nielsen et al., Citation2011). Populations that may particularly benefit from collaborative consortium efforts are those that are relatively small and scattered across Europe. For example, there is a large population of people from Former Yugoslavia in Germany, whereas this population is notably smaller in other European countries (United Nations High Commissioner for Refugees, Citation2001); in such cases, multinational collaborations to validate tests or collect norms may be particularly helpful. Second, this consortium may facilitate the implementation of state-of-the-art knowledge and practices. Third, the ECCroN approach may serve as an example to other regions characterized by high levels of diversity; in fact, some of the instruments developed for diverse individuals in Europe are currently already implemented in other regions, such as the CNTB in Brazil (Araujo et al., Citation2020). Fourth, standardized training at the European level ensures that clinicians across Europe have access to high quality clinical training even where such training is unavailable or not part of the curriculum for neuropsychologists in the individual countries. Last, improvements in and standardization of the measurement of diversity-related variables provides an opportunity to examine theoretical assumptions regarding the influence of these variables on test performance in diverse individuals.
Conclusion
Here, we have raised several important challenges of cross-cultural neuropsychological assessment and assessed the practice landscape for diverse populations in Europe. Furthermore, we provide some solutions to existing barriers for culturally appropriate services. In sum, ECCroN aims to work towards a neuropsychological assessment that is carried out by neuropsychologists trained in cross-cultural assessment, with the help of a well-instructed interpreter where required, and through using tests that are specifically suitable for patients with a wide variety of backgrounds, while taking into account the full spectrum of diversity-related variables in research and clinical practice. Such an approach allows European neuropsychologists to ultimately conduct neuropsychological assessments of diverse individuals that are in line with national professional and ethical codes of conduct (e.g. British Psychological Society, Citation2017; Dansk Psykolog Forening, Citation2001; Nederlands Instituut voor Psychologen, Citation2015). ECCroN will work to build on the momentum of existing partnerships within the collaboration to attract new members from across Europe, establishing measurable impact within the neuropsychology research and practice within Europe and beyond.
Disclosure statement
No potential conflict of interest was reported by the author.
Correction Statement
This article has been republished with minor changes. These changes do not impact the academic content of the article.
Additional information
Funding
References
- Aghvinian, M., Santoro, A. F., Gouse, H., Joska, J. A., Linda, T., Thomas, K. G. F., & Robbins, R. N. (2020). Taking the test: A qualitative analysis of cultural and contextual factors impacting neuropsychological assessment of Xhosa-speaking South Africans. Archives of Clinical Neuropsychology. Advance Online Publication. 36(6), 976–980. https://doi.org/https://doi.org/10.1093/arclin/acaa115
- Agranovich, A. V., Panter, A. T., Puente, A. E., & Touradji, P. (2011). The culture of time in neuropsychological assessment: Exploring the effects of culture-specific time attitudes on timed test performance in Russian and American samples. Journal of the International Neuropsychological Society : JINS, 17(4), 692–701. https://doi.org/https://doi.org/10.1017/S1355617711000592
- Agranovich, A. V., & Puente, A. E. (2007). Do Russian and American normal adults perform similarly on neuropsychological tests? Preliminary findings on the relationship between culture and test performance. Archives of Clinical Neuropsychology : The Official Journal of the National Academy of Neuropsychologists, 22(3), 273–282. https://doi.org/https://doi.org/10.1016/j.acn.2007.01.003
- Al-Jawahiri, F. V., & Nielsen, T. R. (2021). Effects of Acculturation on the Cross-Cultural Neuropsychological Test Battery (CNTB) in a Culturally and Linguistically Diverse Population in Denmark. Archives of Clinical Neuropsychology, 36(3), 381–393. https://doi.org/10.1093/arclin/acz083
- American Psychological Association. (2003). Guidelines on multicultural education, training, research, practice, and organizational change for psychologists. The American Psychologist, 58(5), 377–402. https://doi.org/https://doi.org/10.1037/0003-066X.58.5.377.
- Araujo, N. B., Nielsen, T. R., Barca, M. L., Engedal, K., Marinho, V., Deslandes, A. C., Coutinho, E. S., & Laks, J. (2020). Brazilian version of the European Cross-Cultural Neuropsychological Test Battery (CNTB-BR): Diagnostic accuracy across schooling levels. Revista Brasileira de Psiquiatria (Sao Paulo, Brazil : 1999), 42(3), 286–294. https://doi.org/https://doi.org/10.1590/1516-4446-2019-0539
- Ardila, A. (2005). Cultural values underlying psychometric cognitive testing. Neuropsychology Review, 15(4), 185–195. https://doi.org/https://doi.org/10.1007/s11065-005-9180-y
- Ardila, A. (2007). The impact of culture on neuropsychological test performance. In B. P. Uzzell, M. Ponton, & A. Ardila (Eds.), International handbook of cross-cultural neuropsychology (pp. 23–44). Psychology Press.
- Barker-Collo, S. L. (2001). The 60-item Boston Naming Test: Cultural bias and possible adaptations for New Zealand. Aphasiology, 15(1), 85–92. https://doi.org/https://doi.org/10.1080/02687040042000124
- Birk Jensen, I., & Kolman, L. (2018). Meaning of the EFPA Task Force Cultural and Ethnic Diversity for Psychologists in Europe. In A. Thomas (Ed.), Cultural and ethnic diversity: How European psychologists can meet the challenges (pp. 5–12). Hogrefe.
- British Psychological Society. (2017). BPS practice guidelines. https://www.bps.org.uk/news-and-policy/practice-guidelines
- Byrd, D., Razani, J., Suarez, P., Lafosse, J. M., Manly, J., & Attix, D. K. (2010). Diversity Summit 2008: Challenges in the recruitment and retention of ethnic minorities in neuropsychology. The Clinical Neuropsychologist, 24(8), 1279–1291. https://doi.org/https://doi.org/10.1080/13854046.2010.521769
- Chen, T. B., Lin, C. Y., Lin, K. N., Yeh, Y. C., Chen, W. T., Wang, K. S., & Wang, P. N. (2014). Culture qualitatively but not quantitatively influences performance in the Boston naming test in a chinese-speaking population. Dementia and Geriatric Cognitive Disorders Extra, 4(1), 86–94. https://doi.org/https://doi.org/10.1159/000360695
- CIBG. (2021). Basisberoep en specialisme – specialismen per 1 juni 2021. Ministerie van Volksgezondheid, Welzijn en Sport. https://www.bigregister.nl/over-het-big-register/cijfers/basisberoep-en-specialisme
- Dansk Psykolog Forening. (2001). Retningslinjer for Neuropsykologiske Undersøgelser – specielt med hensyn til test. https://www.dp.dk/neuropsykologiske-undersogelser/
- Daugherty, J. C., Puente, A. E., Fasfous, A. F., Hidalgo-Ruzzante, N., & Perez-Garcia, M. (2017). Diagnostic mistakes of culturally diverse individuals when using North American neuropsychological tests. Applied Neuropsychology. Adult, 24(1), 16–22. https://doi.org/https://doi.org/10.1080/23279095.2015.1036992
- Diez Guardia, N., & Pichelmann, K. (2006). Labour migration patterns in Europe: Recent trends, future challenges. https://ec.europa.eu/economy_finance/publications/pages/publication644_en.pdf
- European Commission/EACEA/Eurydice. (2017). Key Data on Teaching Languages at School in Europe – 2017 Edition Eurydice Report. Publications Office of the European Union.
- Farkas, L. (2017). Data collection in the field of ethnicity: Analysis and comparative review of equality data collection practices in the European Union. Publications Office of the European Union.
- Fasfous, A. F., Peralta-Ramirez, M. I., Perez-Marfil, M. N., Cruz-Quintana, F., Catena-Martinez, A., & Perez-Garcia, M. (2015). Reliability and validity of the Arabic version of the computerized Battery for Neuropsychological Evaluation of Children (BENCI). Child Neuropsychology : A Journal on Normal and Abnormal Development in Childhood and Adolescence, 21(2), 210–224. https://doi.org/https://doi.org/10.1080/09297049.2014.896330
- Fernandez, A. L., & Marcopulos, B. A. (2008). A comparison of normative data for the Trail Making Test from several countries: Equivalence of norms and considerations for interpretation. Scandinavian Journal of Psychology, 49(3), 239–246. https://doi.org/https://doi.org/10.1111/j.1467-9450.2008.00637.x
- Franzen, S., Papma, J. M., van den Berg, E., & Nielsen, T. R. (2020). Cross-cultural neuropsychological assessment in the European Union: A Delphi expert study. Archives of Clinical Neuropsychology. 36(5), 815–830. https://doi.org/https://doi.org/10.1093/arclin/acaa083
- Franzen, S., van den Berg, E., Goudsmit, M., Jurgens, C. K., van de Wiel, L., Kalkisim, Y., Uysal-Bozkir, O., Ayhan, Y., Nielsen, T. R., & Papma, J. M. (2020). A systematic review of neuropsychological tests for the assessment of dementia in non-Western, low-educated or illiterate populations. Journal of the International Neuropsychological Society : JINS, 26(3), 331–351. https://doi.org/https://doi.org/10.1017/S1355617719000894
- Franzen, S., van den Berg, E., Kalkisim, Y., van de Wiel, L., Harkes, M., van Bruchem-Visser, R. L., de Jong, F. J., Jiskoot, L. C., & Papma, J. M. (2019). Assessment of visual association memory in low-educated, non-Western immigrants with the modified Visual Association Test. Dementia and Geriatric Cognitive Disorders, 47(4–6), 345–354. https://doi.org/https://doi.org/10.1159/000501151
- Fujii, D. E. M. (2018). Developing a cultural context for conducting a neuropsychological evaluation with a culturally diverse client: The ECLECTIC framework. The Clinical Neuropsychologist, 32(8), 1356–1392. https://doi.org/https://doi.org/10.1080/13854046.2018.1435826
- Ganguli, M., Chandra, V., Gilby, J. E., Ratcliff, G., Sharma, S. D., Pandav, R., Seaberg, E. C., & Belle, S. (1996). Cognitive test performance in a community-based nondemented elderly sample in rural India: The Indo-U.S. Cross-National Dementia Epidemiology Study. International Psychogeriatrics, 8(4), 507–524. https://doi.org/https://doi.org/10.1017/s1041610296002852
- Glymour, M. M., & Manly, J. J. (2008). Lifecourse social conditions and racial and ethnic patterns of cognitive aging. Neuropsychology Review, 18(3), 223–254. https://doi.org/https://doi.org/10.1007/s11065-008-9064-z
- Goudsmit, M., Uysal-Bozkir, O., Parlevliet, J. L., van Campen, J. P. C. M., de Rooij, S. E., & Schmand, B. (2017). The Cross-Cultural Dementia Screening (CCD): A new neuropsychological screening instrument for dementia in elderly immigrants. Journal of Clinical and Experimental Neuropsychology, 39(2), 163–172. https://doi.org/https://doi.org/10.1080/13803395.2016.1209464
- Health and Social Care Information Centre. (2013). NHS workforce, summary of staff in the NHS: Results from September 2012 census. https://digital.nhs.uk/data-and-information/publications/statistical/nhs-workforce-statistics-overview/nhs-workforce-summary-of-staff-in-the-nhs-results-from-september-2012-census
- Hokkanen, L., Lettner, S., Barbosa, F., Constantinou, M., Harper, L., Kasten, E., Mondini, S., Persson, B., Varako, N., & Hessen, E. (2019). Training models and status of clinical neuropsychologists in Europe: Results of a survey on 30 countries. The Clinical Neuropsychologist, 33(1), 32–56. https://doi.org/https://doi.org/10.1080/13854046.2018.1484169
- Ibanez-Casas, I., Leonard, B. E., Pérez-García, M., & Puente, A. E. (in press). Development of a computerized battery for cross-cultural neuropsychological assessment: The EMBRACED project. Bethlehem University Journal.
- International Test Commission. (2017). The ITC Guidelines for Translating and Adapting Tests (2nd ed.). www.InTestCom.org
- Judd, T., Capetillo, D., Carrión-Baralt, J., Mármol, L., San Miguel-Montes, L., Navarrete, M. G., Puente, A. E., Rodas Romero, H., & Valdés, J., & The NAN Policy and Planning Committee. (2009). Professional Considerations for Improving the NeuropsychologicalEvaluation of Hispanics: A National Academy of Neuropsychology Education Paper. Archives of Clinical Neuropsychology, 24(2), 135–137. https:// https://doi.org/https://doi.org/10.1093/arclin/acp016
- Kunst, A. E., Stronks, K., & Agyemang, C. (2011). Non-communicable diseases. In R. Bernd, P. Mladovsky, W. Devillé, B. Rijks, R. Petrova-Benedict, & M. McKee (Eds.), Migration and Health in the European Union (pp. 101–120). Open University Press.
- Lozano-Ruiz, A., Fasfous, A. F., Ibanez-Casas, I., Cruz-Quintana, F., Perez-Garcia, M., & Pérez-Marfil, M. N. (2021). Cultural bias in intelligence assessment using a culture-free test in Moroccan children. Archives of Clinical Neuropsychology. Advance Online Publication. https://doi.org/https://doi.org/10.1093/arclin/acab005
- Maillet, D., Matharan, F., Le Clesiau, H., Bailon, O., Peres, K., Amieva, H., & Belin, C. (2016). TNI-93: A new memory test for dementia detection in illiterate and low-educated patients. Archives of Clinical Neuropsychology, 31(8), 896–903. https://doi.org/https://doi.org/10.1093/arclin/acw065
- Maillet, D., Narme, P., Amieva, H., Matharan, F., Bailon, O., Le Clesiau, H., & Belin, C. (2017). The TMA-93: A new memory test for Alzheimer’s disease in illiterate and less educated people. American Journal of Alzheimer’s Disease and Other Dementias, 32(8), 461–467. https://doi.org/https://doi.org/10.1177/1533317517722630
- Manly, J. J., & Echemendia, R. J. (2007). Race-specific norms: Using the model of hypertension to understand issues of race, culture, and education in neuropsychology. Archives of Clinical Neuropsychology, 22(3), 319–325. https://doi.org/https://doi.org/10.1016/j.acn.2007.01.006
- Narme, P., Maillet, D., Palisson, J., Le Clesiau, H., Moroni, C., & Belin, C. (2019). How to assess executive functions in a low-educated and multicultural population using a switching verbal fluency test (the TFA-93) in neurodegenerative diseases? American Journal of Alzheimer’s Disease and Other Dementias, 34(7–8), 469–477. https://doi.org/https://doi.org/10.1177/1533317519833844
- Nederlands Instituut voor Psychologen. (2015). Beroepscode voor psychologen 2015. https://www.psynip.nl/en/dutch-association-psychologists/code-of-ethics/code-ethics-2015/respect-general/
- Nielsen, T. R., & Jorgensen, K. (2013). Visuoconstructional abilities in cognitively healthy illiterate Turkish immigrants: A quantitative and qualitative investigation. The Clinical Neuropsychologist, 27(4), 681–692. https://doi.org/https://doi.org/10.1080/13854046.2013.767379
- Nielsen, T. R., Segers, K., Vanderaspoilden, V., Beinhoff, U., Minthon, L., Pissiota, A., Bekkhus-Wetterberg, P., Bjorklof, G. H., Tsolaki, M., Gkioka, M., & Waldemar, G. (2019). Validation of a brief Multicultural Cognitive Examination (MCE) for evaluation of dementia. International Journal of Geriatric Psychiatry, 34(7), 982–989. https://doi.org/https://doi.org/10.1002/gps.5099
- Nielsen, T. R., Segers, K., Vanderaspoilden, V., Bekkhus-Wetterberg, P., Minthon, L., Pissiota, A., Bjorklof, G. H., Beinhoff, U., Tsolaki, M., Gkioka, M., & Waldemar, G. (2018). Performance of middle-aged and elderly European minority and majority populations on a Cross-cultural Neuropsychological Test Battery (CNTB). The Clinical Neuropsychologist, 32(8), 1411–1430. https://doi.org/https://doi.org/10.1080/13854046.2018.1430256
- Nielsen, T. R., Vogel, A., Phung, T. K., Gade, A., & Waldemar, G. (2011). Over- and under-diagnosis of dementia in ethnic minorities: A nationwide register-based study. International Journal of Geriatric Psychiatry, 26(11), 1128–1135. https://doi.org/https://doi.org/10.1002/gps.2650
- Nielsen, T. R., Vogel, A., & Waldemar, G. (2012). Comparison of performance on three neuropsychological tests in healthy Turkish immigrants and Danish elderly. International Psychogeriatrics, 24(9), 1515–1521.
- Rivera Mindt, M., Byrd, D., Saez, P., & Manly, J. (2010). Increasing culturally competent neuropsychological services for ethnic minority populations: A call to action. The Clinical Neuropsychologist, 24(3), 429–453. https://doi.org/https://doi.org/10.1080/13854040903058960
- Scior, K., Gray, J., Halsey, R., & Roth, A. (2007). Selection for clinical psychology training: Is there evidence of any bias towards applicants from ethnic minorities. Clinical Psychology Forum, 175, 7–11.
- Selten, J.-P., Termorshuizen, F., van Sonsbeek, M., Bogers, J., & Schmand, B. (2020). Migration and dementia: A meta-analysis of epidemiological studies in Europe. Psychological Medicine, 51(11), 1838–1845. https://doi.org/https://doi.org/10.1017/S0033291720000586
- Spanish Const. art. 3. (1978).
- Tribe, R., & Thompson, K. (2017). Working with interpreters: Guidelines for psychologists. 1-34. https://www.bps.org.uk/sites/www.bps.org.uk/files/Policy/Policy%20-%20Files/Working%20with%20interpreters%20-%20guidelines%20for%20psychologists.pdf
- United Nations High Commissioner for Refugees. (2001). Asylum applications in industrialized countries: 1980–1999. https://www.unhcr.org/3c3eb40f4.pdf
- van Laer, S. D., Snijder, M. B., Agyemang, C., Peters, R. J., & van den Born, B. H. (2018). Ethnic differences in hypertension prevalence and contributing determinants – The HELIUS study. European Journal of Preventive Cardiology, 25(18), 1914–1922. https://doi.org/https://doi.org/10.1177/2047487318803241
- van Mol, C., & de Valk, H. (2016). Migration and immigrants in Europe: A historical and demographic perspective. In B. Garcés-Mascareñas & R. Penninx (Eds.), Integration Processes and Policies in Europe. IMISCOE Research Series (pp. 31–55). Springer.
- van Oosteren, C., Booi, H., Broekhuizen, J., Cohen, L., de Jong, I., Lindeman, E., Slot, J., & van Vliet, M. (2013). Structurele ontwikkeling Amsterdam. In Een stad voor iedereen (pp. 13–30). Bureau Onderzoek en Statistiek Gemeente Amsterdam.
- Watermeyer, T., & Calia, C. (2019). Neuropsychological assessment in preclinical and prodromal Alzheimer disease: A global perspective. Journal of Global Health, 9(1), 010317. https://doi.org/https://doi.org/10.7189/jogh.09.010317