Co-registration of eye movements and neuroimaging for studying contextual predictions in natural reading

ABSTRACT

Sixteen years ago, Sereno and Rayner (2003. Measuring word recognition in reading: eye movements and event-related potentials. Trends in Cognitive Sciences, 7(11), 489–493) illustrated how “by means of review and comparison” eye movement (EM) and event-related potential (ERP) studies may advance our understanding of visual word recognition. Attempts to simultaneously record EMs and ERPs soon followed. Recently, this co-registration approach has also been transferred to fMRI and oscillatory EEG. With experimental settings close to natural reading, co-registration enables us to directly integrate insights from EM and neuroimaging studies. This should extend current experimental paradigms by moving the field towards studying sentence-level processing including effects of context and parafoveal preview. This article will introduce the basic principles and applications of co-registration and selectively review how this approach may shed light on one of the most controversially discussed issues in reading research, contextual predictions in online language processing.

KEYWORDS:

• Co-registration
• eye movements
• neuroimaging
• fixation-related
• natural reading
• contextual predictions

Acknowledgements

We would like to thank two anonymous reviewers for their insightful and constructive comments, which greatly improved the manuscript.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Florian Hutzler http://orcid.org/0000-0001-8195-4911

Stefan Hawelka http://orcid.org/0000-0002-5304-8521

Notes

1 We would like to point out that the discussion on predictive processing is nuanced and a comprehensive description would be beyond the scope of the article. For an in-depth discussion, we would like to refer the reader to Kuperberg and Jaeger (2016).

2 Word predictability is, traditionally, defined as the proportion [p] of readers (in an independent norming sample) predicting a particular word based on a previous sentence context in a so-called incremental cloze-task (Taylor, 1953). It must be noted that the utilisation of word predictability norms based on this procedure does not come without critique. In particular the circular mapping of behaviour to behaviour has been criticised to constitute a subjective rather than objective measure (Hofmann & Jacobs, 2014, p. 100). Moreover, the rather effortful collection of such predictability norms motivated attempts to approximate word predictability by means of corpus based transitional probabilities (e.g. McDonald & Shillcock, 2003). However, thorough investigations of the relation between effects of transitional probability and word predictability revealed that the variance in the eye movement parameters is better explained by predictability norms (Frisson, Rayner, & Pickering, 2005).

3 Lexical access is defined as the activation of a particular entry in the putative mental lexicon (Inhoff, 1984).

4 Word frequency is defined as the mean prevalence of a word in printed texts (Rayner, 1998, 2009). Please note that in this article we are primarily interested in the emergence of predictability effects in relation to the effect of word frequency. For a detailed analysis of the time-course of other linguistic variables (e.g. word length, orthographic similarity, semantic coherence) we would like to refer the reader to Hauk et al. (2006).

5 Saccade launch site is the takeoff point of a saccade. Saccade launch site distance is the distance between the takeoff point and the beginning of the aimed-at word (Heller & Müller, 1983).

Additional information

Funding

This work was supported by the Austrian Science Fund (FWF; P25799 and P31299). Nicole A. Himmelstoss was financially supported by the Doctoral College “Imaging the Mind” at the University of Salzburg (FWF; W1233).