Why the A/AN prediction effect may be hard to replicate: a rebuttal to Delong, Urbach, and Kutas (2017)

References

• Ardal, S., Donald, M. W., Meuter, R., Muldrew, S., & Luce, M. (1990). Brain responses to semantic incongruity in bilinguals. Brain and Language, 39(2), 187–205. doi: 10.1016/0093-934X(90)90011-5
   PubMed Web of Science ®Google Scholar
• Baayen, H. R., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4), 390–412. doi: 10.1016/j.jml.2007.12.005
   Web of Science ®Google Scholar
• Barr, D. J. (2013). Random effects structure for testing interactions in linear mixed-effects models. Frontiers in Psychology, 4, 328. doi: 10.3389/fpsyg.2013.00328
   PubMed Web of Science ®Google Scholar
• Boekel, W., Wagenmakers, E. J., Belay, L., Verhagen, J., Brown, S., & Forstmann, B. U. (2015). A purely confirmatory replication study of structural brain-behavior correlations. Cortex, 66, 115–133. doi: 10.1016/j.cortex.2014.11.019
   PubMed Web of Science ®Google Scholar
• The British National Corpus. (2007). Distributed by Oxford University Computing Services on behalf of the BNC Consortium. Retrieved from http://www.natcorp.ox.ac.uk/cpr.xml?ID=reference
   Google Scholar
• Bürkner, P.-C. (in press). Brms: An R package for Bayesian multilevel models using Stan. Journal of Statistical Software.
   Google Scholar
• Davies, M. (2008). The corpus of contemporary American English: 450 million words, 1990–present. Retrieved from http://corpus.byu.edu/coca/
   Google Scholar
• Dell, G. S., & Chang, F. (2014). The P-chain: Relating sentence production and its disorders to comprehension and acquisition. Philosophical Transactions of the Royal Society B, 369, 20120394. doi: 10.1098/rstb.2012.0394
   PubMed Web of Science ®Google Scholar
• DeLong, K. A., Groppe, D. M., Urbach, T. P., & Kutas, M. (2012). Thinking ahead or not? Natural aging and anticipation during reading. Brain and Language, 121(3), 226–239. doi: 10.1016/j.bandl.2012.02.006
   PubMed Web of Science ®Google Scholar
• DeLong, K. A., & Kutas, M. (2016). Hemispheric differences and similarities in comprehending more and less predictable sentences. Neuropsychologia, 91, 380–393. doi: 10.1016/j.neuropsychologia.2016.09.004
   PubMed Web of Science ®Google Scholar
• DeLong, K. A., Quante, L., & Kutas, M. (2014). Predictability, plausibility, and two late ERP positivities during written sentence comprehension. Neuropsychologia, 61(1), 150–162. doi: 10.1016/j.neuropsychologia.2014.06.016
   PubMed Web of Science ®Google Scholar
• DeLong, K. A., Urbach, T. P., & Kutas, M. (2005). Probabilistic word pre-activation during language comprehension inferred from electrical brain activity. Nature Neuroscience, 8(8), 1117–1121. doi: 10.1038/nn1504
   PubMed Web of Science ®Google Scholar
• DeLong, K. A., Urbach, T. P., & Kutas, M. (2017). Is there a replication crisis? Perhaps. Is this an example? No: A commentary on Ito, Martin & Nieuwland (2016). Language, Cognition, and Neuroscience. doi: 10.1080/23273798.2017.1279339
   PubMed Web of Science ®Google Scholar
• Foucart, A., Martin, C. D., Moreno, E. M., & Costa, A. (2014). Can bilinguals see it coming? Word anticipation in L2 sentence reading. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(5), 1461–1469. doi: 10.1037/a0036756
   PubMed Web of Science ®Google Scholar
• Gelman, A. (2015). The connection between varying treatment effects and the crisis of unreplicable research: A Bayesian perspective. Journal of Management, 41(2), 632–643. doi: 10.1177/0149206314525208
   Web of Science ®Google Scholar
• Hopp, H. (2009). The syntax-discourse interface in near-native L2 acquisition: Off-line and on-line performance. Bilingualism: Language and Cognition, 12(4), 463–483. doi: 10.1017/S1366728909990253
   Web of Science ®Google Scholar
• Huettig, F. (2015). Four central questions about prediction in language processing. Brain Research, 1626, 118–135. doi: 10.1016/j.brainres.2015.02.014
   PubMed Web of Science ®Google Scholar
• Ito, A., Corley, M., Pickering, M. J., Martin, A. E., & Nieuwland, M. S. (2016). Predicting form and meaning: Evidence from brain potentials. Journal of Memory and Language, 86, 157–171. doi: 10.1016/j.jml.2015.10.007
   Web of Science ®Google Scholar
• Ito, A., Martin, A. E., & Nieuwland, M. S. (2016a). How robust are prediction effects in language comprehension? Failure to replicate article-elicited N400 effects. Language, Cognition and Neuroscience. doi: 10.1080/23273798.2016.1242761
   Google Scholar
• Ito, A., Martin, A. E., & Nieuwland, M. S. (2016b). On predicting form and meaning in a second language. Journal of Experimental Psychology: Learning Memory and Cognition, doi: 10.1037/xlm0000315
   PubMed Web of Science ®Google Scholar
• Jäger, L. A., Engelmann, F., & Vasishth, S. (2017). Similarity-based interference in sentence comprehension: Literature review and Bayesian meta-analysis. Journal of Memory and Language, 94, 316–339. doi: 10.1016/j.jml.2017.01.004
   Web of Science ®Google Scholar
• Jeffreys, H. (1961). The theory of probability. Oxford: Oxford University Press.
   Google Scholar
• Kuperberg, G. R., & Jaeger, T. F. (2016). What do we mean by prediction in language comprehension? Language Cognition & Neuroscience, 31(1), 32–59. doi: 10.1080/23273798.2015.1102299
   PubMed Web of Science ®Google Scholar
• Martin, C. D., Thierry, G., Kuipers, J. R., Boutonnet, B., Foucart, A., & Costa, A. (2013). Bilinguals reading in their second language do not predict upcoming words as native readers do. Journal of Memory and Language, 69(4), 574–588. doi: 10.1016/j.jml.2013.08.001
   Web of Science ®Google Scholar
• Masson, M. E. J. (2011). A tutorial on a practical Bayesian alternative to null-hypothesis significance testing. Behavior Research Methods, 43, 679–690. doi: 10.3758/s13428-010-0049-5
   PubMed Web of Science ®Google Scholar
• Moreno, E. M., & Kutas, M. (2005). Processing semantic anomalies in two languages: An electrophysiological exploration in both languages of Spanish-English bilinguals. Cognitive Brain Research, 22(2), 205–220. doi: 10.1016/j.cogbrainres.2004.08.010
   PubMedGoogle Scholar
• Nieuwland, M. S. (2016). Quantification, prediction and the online impact of sentence truth-value: Evidence from event-related potentials. Journal of Experimental Psychology: Learning, Memory, and Cognition, 42(2), 316–334. doi: 10.1037/xlm0000173
   PubMed Web of Science ®Google Scholar
• Nieuwland, M. S., Politzer-Ahles, S., Heyselaar, E., Segaert, K., Darley, E., Kazanina, N., … Huettig, F. (2017). Limits on prediction in language comprehension: A multi-lab failure to replicate evidence for probabilistic pre-activation of phonology. BioRxiv, 111807. doi: 10.1101/111807
   Google Scholar
• Otten, M., Nieuwland, M. S., & Van Berkum, J. J. A. (2007). Great expectations: Specific lexical anticipation influences the processing of spoken language. BMC Neuroscience, 8, 89. doi: 10.1186/1471-2202-8-89
   PubMed Web of Science ®Google Scholar
• Otten, M., & Van Berkum, J. J. A. (2009). Does working memory capacity affect the ability to predict upcoming words in discourse? Brain Research, 1291, 92–101. doi: 10.1016/j.brainres.2009.07.042
   PubMed Web of Science ®Google Scholar
• Pashler, H., & Harris, C. R. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7(6), 531–536. doi: 10.1177/1745691612463401
   Web of Science ®Google Scholar
• Pickering, M. J., & Garrod, S. (2013). An integrated theory of language production and comprehension. Behavioural and Brain Sciences, 36, 329–392. doi: 10.1017/S0140525X12001495
   PubMed Web of Science ®Google Scholar
• Raftery, A. E. (1995). Bayesian model selection in social research. In P. V. Marsden (Ed.), Sociological methodology (pp. 111–163). Cambridge: Blackwell.
   Google Scholar
• Rommers, J., Meyer, A. S., & Huettig, F. (2013). Object shape and orientation do not routinely influence performance during language processing. Psychological Science, 24(11), 2218–2225. doi: 10.1177/0956797613490746
   PubMed Web of Science ®Google Scholar
• Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16(2), 225–237. doi: 10.3758/PBR.16.2.225
   PubMed Web of Science ®Google Scholar
• Urbach, T. P., & Kutas, M. (2010). Quantifiers more or less quantify on-line: ERP evidence for partial incremental interpretation. Journal of Memory and Language, 63(2), 158–179. doi: 10.1016/j.jml.2010.03.008
   PubMed Web of Science ®Google Scholar
• Van Berkum, J. J. A., Brown, C. M., Zwitserlood, P., Kooijman, V., & Hagoort, P. (2005). Anticipating upcoming words in discourse: Evidence from ERPs and reading times. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(3), 443–467. doi: 10.1037/0278-7393.31.3.443
   PubMed Web of Science ®Google Scholar
• Van Petten, C., & Luka, B. J. (2012). Prediction during language comprehension: Benefits, costs, and ERP components. International Journal of Psychophysiology, 83(2), 176–190. doi: 10.1016/j.ijpsycho.2011.09.015
   PubMed Web of Science ®Google Scholar
• Verhagen, J., & Wagenmakers, E.-J. (2014). Bayesian tests to quantify the result of a replication attempt. Journal of Experimental Psychology: General, 143(4), 1457–1475. doi: 10.1037/a0036731
   PubMed Web of Science ®Google Scholar
• Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14(5), 779–804. doi: 10.3758/BF03194105
   PubMed Web of Science ®Google Scholar
• Weber-Fox, C. M., & Neville, H. J. (1996). Maturational constraints on functional specializations for language processing: ERP and behavioral evidence in bilingual speakers. Journal of Cognitive Neuroscience, 8(3), 231–256. doi: 10.1162/jocn.1996.8.3.231
   PubMed Web of Science ®Google Scholar
• Wicha, N. Y. Y., Bates, E. A., Moreno, E. M., & Kutas, M. (2003). Potato not Pope: Human brain potentials to gender expectation and agreement in Spanish spoken sentences. Neuroscience Letters, 346(3), 165–168. doi: 10.1016/S0304-3940(03)00599-8
   PubMed Web of Science ®Google Scholar
• Wicha, N. Y. Y., Moreno, E. M., & Kutas, M. (2012). Anticipating words and their gender: An event-related brain potential study of semantic integration, gender expectancy, and gender agreement in Spanish sentence reading. Cognitive Neuroscience, 16(7), 1272–1288. doi: 10.1162/0898929041920487
   Web of Science ®Google Scholar
• Zwaan, R. A., & Pecher, D. (2012). Revisiting mental simulation in language comprehension: Six replication attempts. PLoS ONE, 7(12), e51382. doi: 10.1371/journal.pone.0051382
   PubMed Web of Science ®Google Scholar