Unresolved problems with the “I”, the “A”, and the “T”: A logical and psychometric critique of the Implicit Association Test (IAT)

Abstract

The Implicit Association Test (IAT) had already gained the status of a prominent assessment procedure before its psychometric properties and underlying task structure were understood. The present critique addresses five major problems that arise when the IAT is used for diagnostic inferences: (1) the asymmetry of causal and diagnostic inferences; (2) the viability of the underlying association model; (3) the lack of a testable model underlying IAT-based inferences; (4) the difficulties of interpreting difference scores; and (5) the susceptibility of the IAT to deliberate faking and strategic processing. Based on a theoretical reflection of these issues, and a comprehensive survey of published IAT studies, it is concluded that a number of uncontrolled factors can produce (or reduce) significant IAT scores independently of the personality attribute that is supposed to be captured by the IAT procedure.

Acknowledgments

The research underlying this chapter was supported by grants of the German Science Foundation (Deutsche Forschungsgemeinschaft) given to the first two authors. The authors gratefully acknowledge helpful comments on a draft of this paper by Henning Plessner and Jan De Houwer. The coding of the IAT studies was accomplished by Olivia Balmer, Julia Binder, Isabelle Dreyfus, Christoph Schmitz, and also by Sven-Yves Renn, who tragically died at the beginning of a promising scientific career.

Notes

¹For example, the argument, presented by a prominent IAT proponent in a review of a former version of this chapter, that “the IAT is not a diagnostic instrument but a tool for measuring individual differences”, reflects a misunderstanding of the term “diagnostic”.

²It is unclear whether the term implicit in the IAT refers to implicitly learned attitudes or associations, to implicitly assessed attitudes or associations, or to a construct of implicit associations, which would presuppose the existence of explicit associations by complement.

³The two conditionals actually differ by the factor 10 (cf. Gigerenzer & Hoffrage, 1995).

⁴ Table 1 includes φ correlations, for all combinations of p(IAT+) and p(A | IAT+), calculated from cell frequencies a = p(IAT+) · p(A | IAT+); b = p(IAT – ) · p(A | IAT – ); c = 1 – a; and d = 1 – b. It was assumed that b = 0, because A should never occur in the absence of IAT+.

⁵For example, the correlation between race and education may be strong across districts but negligible at individual level. The reason is that the two levels, districts and individuals, are subject to completely different influences (e.g., base rates of socio-economic level affecting districts; learning motivation affecting individuals). Boundary conditions for relying on group data are developed in Hammond (1973).

⁶Even when Rothermund and Wentura did not call their principle “strategic”, their findings clearly fit the present definition of strategic influences.

⁷We searched in the PsycINFO database for peer-reviewed articles, scanning keywords, titles, and abstracts for “IAT”, “implicit association”, “implicit attitude”, “implicit stereotype”, “implicit evaluation”, “implicit measure”, “automatic association”, “automatic attitude”, and “automatic stereotype”. After omitting false alarms (e.g., IAT related to “Intervention Assistance Teams” or “intracarotid amobarbital test”), we identified 200 IAT-related, peer-reviewed articles in October 2005, published in the English language, from 1998 up to 2005). Of these, 34 articles were concerned with IAT derivates like the Extrinsic Affective Simon Task (EAST) or merely discussed the IAT in a review, editorial, comment, or essay, leaving 166 empirical articles comprising 331 studies that dealt with 495 separate IAT applications (cf. Appendix and see References marked with an asterisk).

⁸In IAT studies using the individual “self” and “others” as categories, a pretest for both target and attribute stimuli on cross-category associations does not make sense (cf. “self” in the Appendix). Note also that, depending on a researcher's focus, “self” can sometimes be used as a target category or as an attribute category.