Generally representative is representative of none: commentary on the pitfalls of IQ test standardization in multicultural settings

Abstract

Objective: The aim of this paper is to address the issue of IQ testing within the multicultural context, with a focus on the adequacy of nationwide population-based norms vs. demographically stratified within-group norms for valid assessment purposes. Burgeoning cultural diversity worldwide creates a pressing need to cultivate culturally fair psychological assessment practices. Method: Commentary is provided to highlight sources of test-taking bias on tests of intellectual ability that may incur invalid placement and diagnostic decisions in multicultural settings. Methodological aspects of population vs. within-group norming solutions are delineated and the challenges of culturally relevant norm development are discussed. Illustrative South African within-group comparative data are supplied to support the review. A critical evaluation of the South African WAIS-III and the WAIS-IV standardizations further serves to exemplify the issues. Results: A flaw in both South African standardizations is failure to differentiate between African first language individuals with a background of advantaged education vs. those from educationally disadvantaged settings. In addition, the standardizations merge the performance outcomes of distinct racial/ethnic groups that are characterized by differentially advantaged or disadvantaged backgrounds. Consequently, the conversion tables are without relevance for any one of the disparate South African cultural groups. Conclusions: It is proposed that the traditional notion of a countrywide unitary norming (also known as ‘population-based norms’) of an IQ test is an unsatisfactory model for valid assessment practices in diverse cultural contexts. The challenge is to develop new solutions incorporating data from finely stratified within-group norms that serve to reveal rather than obscure cross-cultural disparity in cognitive test performance.

Keywords:

• Multicultural
• cross-cultural
• normative data
• standardization
• IQ testing
• WAIS-III
• WAIS-IV
• South Africa

This article is referred to by:

A commentary on ‘generally representative is representative of none: pitfalls of IQ test standardization in multicultural settings’ by A.B. Shuttleworth-Edwards

Disclosure statement

The author derives no financial gain (i.e. royalties) from any of the cited book chapters. From time to time, the author has accepted invitations to present workshops on cross-cultural issues at conferences and professional meetings, or for training purposes, at which the South African Wechsler research data cited in this article have been used for illustrative purposes, together with collations of local normative data for other commonly employed cognitive tests. In training contexts, a small honorarium of around R6000–R8000 (300–500 USD) has normally been awarded.

Notes

1. All book chapters cited in this article were subjected to peer-review, thereby qualifying as accredited academic outputs for university subsidy purposes in the year of their publication, by the South African government Department of Higher Education.

2. There have been three South African Wechsler test standardizations including the South African WAIS (1969); the South African WAIS-III (Claassen et al., 2001); and the South African WAIS-IV (Wechsler, 2014). For comparative coherence in the present commentary, the South African standardization of the WAIS-III (for which there is no official acronym) is descriptively abbreviated to the SA WAIS-III; the South African standardization of the WAIS-IV (for which the official acronym is the WAIS-IV^SA) is descriptively abbreviated to the SA WAIS-IV. The use of these abbreviations serves to distinguish them from each other, and in turn to differentiate them from the United States standardizations of the WAIS-III and WAIS-IV versions that will be referred to as the U.S. WAIS-III and the U.S. WAIS-IV, respectively.

3. The terms black, colored, Indian or Indian/Asian, and white are used in this article with reference to the four main subgroups that make up the South African population as applied in the most recent 2011 South African census (2012). The majority of the South African population is made up of black African individuals (approximately 79% of the population), who have one of the indigenous African languages as their first language. The remainder of the population is made up of colored, Indian/Asian, and white individuals (9, 3 and 9% of the population, respectively), who have either Afrikaans or English as their first language. While the term ‘colored’ is viewed as derogatory in some parts of the world, this is not the case in South Africa, where it is the generally accepted nomenclature used to differentiate an important mixed-race segment of the population who predominantly live in the Western Cape, speaking either Afrikaans and English rather than any of the indigenous African languages. The group is made up of individuals descended from the intermarriage of white settlers, African natives, and Asian slaves who were brought to South Africa from the Dutch colonies of Asia in the eighteenth and nineteenth centuries.

The motivation to distance oneself from the concept of race because of noxious discriminatory practices is compelling (Bowman, Seedat, Duncan, & Burrows, 2006; Van Ommen, 2013). However, race continues to be the officially employed term to demarcate the population groups in the South African arena for policy purposes, with an important aim of being corrective of historically embedded discriminatory practices (for example, affirmative action). Similarly, there are cogent reasons not to deny or obfuscate race as a socially relevant concept in normative data research, due to sociocultural effects that have been, and continue to be inextricably linked with race that impact on psychometric test-taking ability (Van Ommen, 2013).

4. Importantly, the recommendation is to use the U.S. WAIS-III version for educationally advantaged individuals regardless of their race or language of origin means using that test in its entirety, which is the same method employed for the preliminary within-group norming data of Shuttleworth-Edwards et al. (2004) and Shuttleworth-Edwards, Gaylard et al. (2013). The recommendation does not imply using the SA WAIS-III version of the test in conjunction with the U.S. WAIS-III norms. Similarly, the recommendation to use the U.S. WAIS-IV for educationally advantaged individuals regardless of their race or language of origin means using that test in its entirety, and does not imply using the SA WAIS-IV version of the test in conjunction with the U.S. WAIS-IV norms.

5. An overall inflationary trend on the SA WAIS-IV compared with the U.S. standardization is estimated to lie somewhere between 15 and 23 IQ points. The approximation is achieved by calculating the overall IQ test scores from the core subtests using the SA WAIS-IV scoring manual, in association with an increase of 2 scaled score points on each subtest over the U.S. average of 10 (yielding an IQ of 115), or an increase of 3 scaled score points over the U.S. average of 10 on each subtest (yielding an IQ of 123). The purpose of the commentary is broadly to illustrate the presence of a substantive inflationary trend on the South African standardization that for the most part is either 2 or 3 scaled score points per subtest. This is likely to be a conservative estimate in that there are invariably several subtest scaled scores inflated by three points, and in some instances there is inflation up to as much as four scaled score points. For instance, as indicated in the text, for the 50–59.11 age group the obtained scaled scores of 14 for both Coding and Symbol Search convert into a PSI of 126, i.e. an increase of 26 Index score points over the U.S. standardization. The exact extent of the inflation for every subtest, Index and IQ score per age grouping will require intricate statistical analysis beyond the scope of the present paper that is yet to be formally calculated, complicated by the fact that the SA WAIS-IV and U.S. WAIS-IV older age groupings are not equivalent.