How robust are prediction effects in language comprehension? Failure to replicate article-elicited N400 effects

ABSTRACT

Current psycholinguistic theory proffers prediction as a central, explanatory mechanism in language processing. However, widely-replicated prediction effects may not mean that prediction is necessary in language processing. As a case in point, C. D. Martin et al. [2013. Bilinguals reading in their second language do not predict upcoming words as native readers do. Journal of Memory and Language, 69(4), 574–588. doi:10.1016/j.jml.2013.08.001] reported ERP evidence for prediction in native- but not in non-native speakers. Articles mismatching an expected noun elicited larger negativity in the N400 time window compared to articles matching the expected noun in native speakers only. We attempted to replicate these findings, but found no evidence for prediction irrespective of language nativeness. We argue that pre-activation of phonological form of upcoming nouns, as evidenced in article-elicited effects, may not be a robust phenomenon. A view of prediction as a necessary computation in language comprehension must be re-evaluated.

KEYWORDS:

• Prediction
• language comprehension
• ERP
• N400
• bilingualism

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCiD

Aine Ito http://orcid.org/0000-0003-4408-8801

Notes

1. Martin et al. argued that the article effect constituted an N400 effect, but their effect did not have the posterior distribution typically observed for N400 effects. At parietal channels, where N400 differences are usually maximal, no difference was observed. Moreover, their N400 distribution was distorted by the reference procedure, because they used a common average reference rather than the average-mastoid procedure typically used in N400 research. Had they used the average-mastoid procedure, they could have observed a larger N400 for expected articles than for unexpected articles, which would be inconsistent with their interpretation.

2. Cloze probability of a word is established in a cloze test as the proportion of participants who used that word to complete a given sentence fragment. It is commonly used as a proxy for predictability/expectedness.

3. There are also differences in terms of data analysis. For example, whereas DeLong et al. showed graded effects of predictability by reporting correlation analysis of the N400 waveform and the cloze value of the article, Martin et al. analysed their data using a factorial design with the factor expected/unexpected. In addition, DeLong et al. used a mastoid reference and filtered the data offline using 0.2–15 Hz filter, while Martin et al. used a global reference and filtered the data offline using a 30 Hz low-pass filter. We will discuss further these differences in the results section when relevant to our own results.

4. Martin et al. reported that the SOA in their study was 500 ms, but it was actually 700 ms (as confirmed in a personal communication by Martin).

5. We initially used 500 ms SOA as was reported in Martin et al., but also used 700 ms SOA as this was the actual SOA used in their study.

6. Following Martin et al., we also conducted another cloze test with different participants (native N = 24, non-native N = 12), in which the sentences were truncated before the noun and always presented with an unexpected article (e.g. “As it’'s rainy, it’'s better to go out with a … ”). Participants were reminded that sentences always ended with “a” or “an”, and they were instructed to take this, as well as the sentence content, into account when choosing nouns that fit in the context. In this cloze test, the mean native cloze probability was 16.8% (SD = 8.6) for expected nouns and 31.7% (SD = 23.5) for unexpected nouns, and the mean non-native cloze probability was 12.6% (SD = 10.0) for expected nouns and 30.5% (SD = 29.7) for unexpected nouns. In Martin et al., the mean non-native cloze probability was 3.5% for expected nouns and 37.4% for unexpected nouns. The non-native cloze probability for expected nouns was higher in our study, but the pattern of results was similar.

7. Martin et al. used a global reference instead of a mastoid reference. The global reference procedure is uncommon and sub-optimal in N400 research because the N400 can be broadly distributed across the scalp, in which case subtracting the average of all channels may diminish the observed effects. The global reference procedure may also lead to very different scalp distribution effects compared to the mastoid reference procedure. In order to explore potential effects of the different reference, we also report results after using the global reference.

8. We followed a reviewer's suggestion and explored whether we would obtain an article-elicited N400 effect in other time windows. This additional analysis used the same ANOVAs run successive 50 ms time bins from 200 ms until 500 ms (200–250, 250–300, and so on), with a Bonferroni correction for the multiple comparisons. None of the time bins showed any significant effect of expectedness, p > .1.

Additional information

Funding

AEM was supported by a Future Research Leaders Grant from the Economic and Social Research Council of the United Kingdom [ES/K009095/1].