Implications of measures of reliability for theories of priming: Activity in semantic memory is inherently noisy and uncoordinated

Abstract

In experiments on semantic priming, participants vary substantially in the absolute magnitude of priming they produce. Are these individual differences systematic or do they arise from random processes, and does the answer carry theoretical implications? To find out, we examined split‐half and test–retest reliability of semantic priming in a series of two‐session experiments that crossed relatedness proportion (RP, at .25, .50, and .75) with stimulus–onset asynchrony (SOA, at 200, 350, and 800 ms). Low reliability would indicate little coherence of activity within semantic memory, so that the degree to which any given association influences performance is uncorrelated from one association and time of testing to the next. High reliability would indicate that each person tends to harness his or her semantic knowledge consistently, applying semantic relations among words in much the same way from one word to the next and one time of testing to the next to help task performance, resulting in systematic individual differences. What we observed was low reliability—often zero. When conditions highlighted “automatic activation” (low RP, short SOA), priming was completely uncorrelated from item to item and session to session. Both split‐half and test–retest reliability increased to significant levels under conditions that raised the probability that an activated or retrieved episode of prime experience would help with target recognition, and the probability of intentionally generating the target from the prime. Thus, task‐relevant" utility imposes a modicum of order on semantic associations that are otherwise noisy and uncoordinated. Harnessing semantic memory is like herding cats—without considerable constraint, associations tend to come and go their own ways in independent fashion.

Notes

Please address all correspondence to Jennifer A. Stolz, Department of Psychology, University of Waterloo, Waterloo, Ontario, Canada N2L 3G1. Email: [email protected]

This work was supported by grants from the Natural Sciences and Engineering Research Council of Canada to JAS and to DB and was presented at the 41st annual meeting of the Psychonomic Society.

We thank Theeus Devitt‐Carolan, Danna Ingleton, and Agnes Polanowski for data collection.

We note that the term “conditional automaticity” is theoretically useful only to the extent that the conditions required to support said process are stated a priori. Furthermore, as soon as one concedes that a process is conditional, the idea that it is also automatic loses some of its force. Specifically, if a process is said to be automatic given that conditions x and y are met, it must be shown to be the case that other processes, said not to be automatic in that task environment, do not occur when the same conditions x and y are met.

Throughout this paper, we report Cohen's (1988) f (or the related f ²) for our correlation tests as a measure of effect size. Cohen defines f = .10 as a small effect size, f = .25 as a medium effect size, and f = .40 as a large effect size (pp. 285–287). Likewise, f ² = .02 represents a small effect size, f ² = .15 represents a medium effect size, and f ² = .35 represents a large effect size.

Although the omnibus RP × Priming interaction was not significant, the magnitude of the priming effect appeared to be larger at RP = .75 (36 ms) than at the lower RPs (28 ms and 29 ms for RP = .25 and .50, respectively). To test whether this difference was significant, we compared the pooled priming effects for the RP = .25 and .50 conditions to that produced by the RP = .75 group. This difference was not reliable, F(1, 238) = 2.52, MSe = 851.28, p = .114. Our general conclusion is that although there may have been a tendency for at least some participants to utilize a strategy such as expectancy to enhance priming effects, there is no evidence that it played a major role at this short, 200 ms, SOA.

Once again, we compared the magnitude of priming effects pooled from the RP = .25 and .50 groups to that produced by the RP = .75 group. Although there was a tendency for priming to be larger in the RP = .75 group, this was not significant, F(1, 234) = 2.88, MSe = 712.70, p = .091.