Balancing exogenous and endogenous cortical rhythms for speech and language requires a lot of entraining: a commentary on Meyer, Sun & Martin (2020)

This article refers to:

Synchronous, but not entrained: exogenous and endogenous cortical rhythms of speech and language processing

One of the most fruitful areas of research in recent years in the cognitive neuroscience of language has investigated the role of neural entrainment in speech segmentation and comprehension (e.g. Giraud & Poeppel, 2012; Kösem & Van Wassenhove, 2017; Meyer, 2018). Another line of research has probed the role of neural synchronisation in the processing of words, phrases, sentences, discourse and event structures, and even pragmatics (e.g. Bastiaansen & Hagoort, 2006; Lewis, Wang, & Bastiaansen, 2015; Prystauka & Lewis, 2019; Weiss & Mueller, 2012). Traditionally these two lines of enquiry have foraged independently, each gathering new observations and new insights, and independently mapping out the dense forests that constitute our knowledge and understanding of how the brain gives rise to what is arguably our most uniquely human characteristic. Much progress has been made, and the time appears ripe to compare maps and try to discover common paths, clearings that seemed obvious to one but perhaps not the other line of enquiry, and tangled thickets that might best be explored in tandem.

In a recent article, Meyer, Sun, and Martin (2019; henceforth referred to as MSM) make a case for precisely such a synthesis between these two lines of research. MSM make the important distinction between what they refer to as entrainment proper and internal synchronicity. The key distinguishing feature being that entrainment proper requires an external (quasi-)rhythmic stimulus that drives cortical synchronicity at rates corresponding to rhythmic characteristics of the driving stimulus. Internal synchronicity, on the other hand, is endogenously generated based on abstract linguistic representations that allow for inferences about linguistic information at various levels of abstraction (e.g. words, phrases, sentences etc.). Crucially, such synchronicity does not require these linguistic units to be marked in the physical characteristics of the speech stimulus.

From my perspective, MSM offer two major insights that have the potential to drive the field forward and to open new avenues of research. First, they suggest that much of what has been called entrainment in the literature might turn out to be internal synchronicity in disguise. The crux of their argument is that it is difficult to tell the two apart because the rates at which entrainment is observed very often overlap with the rates expected for internal synchronicity. MSM discuss a number of studies in the entrainment literature for which this is at least a plausible alternative explanation. A second important insight is MSM’s suggestion that entrainment proper and internal synchronicity may provide a degree of useful redundancy when it comes to speech segmentation and/or comprehension. The key suggestion is that these mechanisms may complement one another to bring about sustained entrainment when either mechanism alone may not be sufficient to allow for decoding the relevant information from the speech signal. Speech is quasi-rhythmic for example and often does not provide sufficient information in the physical stimulus to allow for entrainment proper. In such cases, internal synchronicity can provide a helping hand by “filling in the blanks” so to say; generating entrainment based on inferences from abstract symbolic representations about the expected rhythmicity of different types of linguistic information. MSM propose that both entrainment proper and internal synchronisation might be consistently at work during speech segmentation and comprehension, and that the relative weighting of these two factors may be modified appropriately depending on the situation.

I find it difficult to quibble with MSM and consider their proposal(s) to be very much in the spirit of recent thinking about how best to reconcile the literatures on entrainment and network synchronisation (Lakatos, Gross, & Thut, 2019). In the remainder of this commentary, I outline a few considerations that I think will be important to bear in mind while navigating some of the new paths opened up by MSM, and I speculate about some intriguing consequences in case MSM’s proposal turns out to be on the right track.

Meaningful measurements make meaningful distinctions

An important technical point that should be considered is the method used to quantify entrainment and/or synchronicity. For many, specifics of such technical details can often be dry, uninteresting, and have little influence on interpretation of the overall findings. In the particular case at hand, however, some of the specifics are immensely important for the kinds of conclusions one can draw from the findings. Some studies, for instance, have employed explicit measures of coupling (using measures like coherence or mutual information) between physical characteristics of the speech signal (e.g. amplitude envelope, syllable rate, phoneme rate, etc.), and frequency-specific spectral information (e.g. power, phase) in the Magnetoencephalography/Electroencephalography (M/EEG) data concurrently recorded (e.g. Ahissar et al., 2001; Gross et al., 2013; Keitel, Gross, & Kayser, 2018; Lam, Hultén, Hagoort, & Schoffelen, 2018). Others have instead measured M/EEG while people listen to speech with known spectral characteristics centred at specific rates of interest (e.g. syllable rate, phoneme rate, etc.), and looked for entrainment by searching for corresponding peaks in M/EEG power spectra, or in measures of phase consistency across trials (e.g. Ding, Melloni, Zhang, Tian, & Poeppel, 2016; Kösem et al., 2018; Meyer, Henry, Gaston, Schmuck, & Friederici, 2017). Importantly, in the latter cases entrainment is inferred without explicitly computing a measure of coupling between the speech signal and the neural activity. Both approaches have merits and are typically an appropriate choice for the question of interest in the studies in which they are employed. When synthesising the wider literature, however, the details are frequently glossed over and reviews will often treat both approaches as indicative of entrainment proper, when in fact only the former explicitly establishes a relationship with the external driving speech stimulus. A crucial corollary is that if one is expecting to observe internal synchronicity, then it does not make much sense to use measures that look for coupling between the speech stimulus and frequency-specific neural activity. This is because in such cases, the rates at which synchronicity is expected are internally generated based on inferences from abstract linguistic information, and should thus not (necessarily) be expected to have physically observable signatures in the speech signal itself.

What’s the going rate and where does it originate?

In characterising the anatomical implementation of the endogenous-exogenous balance (relative weighting of internal synchronicity and entrainment proper) MSM suggest that entrainment proper might be restricted to primary sensory (in this case auditory) cortices. As the information generating the synchronicity becomes more abstract and categorical in nature, the balance shifts towards internal synchronicity and away from entrainment proper. MSM propose that this shift may be reflected in internal synchronicity moving out from auditory association cortices and spreading to more widely distributed brain networks. This raises the important question of which brain regions will be incorporated into these more widespread networks related to the processing of different kinds of abstract information. Similarly, one might ask whether such networks are mediated by internal synchronicity at different rates depending on the type of information or level of abstraction of the associated inferences. Alternatively, the observed internal synchronicity may be related to network size (Buzsáki, Logothetis, & Singer, 2013; Lakatos et al., 2019) rather than the type of linguistic information with which it is associated. Speculatively, one might imagine that there is some correlation between the degree of abstraction of linguistic information, the size of the neural networks expressing associated internal synchronicity, and the rate at which such networks express said synchronicity. This offers an intriguing hypothesis to explore in future experimental work.

A related question one might ask is how internal synchronicity in widespread networks influences entrainment proper. Should we expect internal synchronicity to exert its influence by matching the delta and/or theta rate typically observed for entrainment proper in auditory cortices? Alternatively, might it be the case that internal synchronicity in widespread brain networks modulates entrainment proper in sensory cortices by resetting or aligning the phase of oscillations in those regions, perhaps through a mechanism like cross-frequency coupling (e.g. Jensen & Colgin, 2007)? Working out precisely how (or even whether) internal synchronicity interacts with entrainment proper in the service of better aligning information uptake from the speech input with top-down inferences about more abstract linguistic representations, will be a fascinating direction to pursue in future experimental and computational modelling work.

A role for control?

Perhaps the most appealing aspect of models that incorporate entrainment for the purposes of speech segmentation (Ghitza, 2011; Giraud & Poeppel, 2012) is that the processing is largely automatic. In such models, auditory cortices entrain to the relevant rates (e.g. syllabic or phonemic) for speech segmentation, simply in virtue of being confronted with (quasi-)rhythmic speech input. The entire process is dynamic and self-organising (Giraud & Poeppel, 2012; Obleser & Kayser, 2019), and the auditory system is thought to be anatomically and physiologically predisposed (likely due to a combination of evolutionary history and statistical learning across development) to entrain more to speech input than to other auditory inputs, and to do so at specific frequencies corresponding to the relevant rates at which speech segmentation typically unfolds within a language. This elegant proposal allows for an explanation of speech segmentation that does not require intervention by some top-down monitoring system to keep track of whether the input constitutes speech or not.

The quasi-rhythmic nature of relevant characteristics of the speech signal to which the auditory system is expected to entrain has posed a niggling question to this otherwise elegant theory of speech segmentation by the brain’s auditory system. MSM’s proposal that a fundamental part of the dynamics of the system is a tradeoff in the relative weighting of entrainment proper and internal synchronicity, is itself a very elegant way to deal with this observation that the speech signal is not perfectly rhythmic. The possibility that in cases where rhythmicity in the speech signal is not perfectly isochronous internal synchronicity is weighted more highly and essentially “fills in the blanks” for driving the entrainment of auditory cortices, provides an appealing answer to such questions.

One important question that arises, however, is precisely how the weighting between intrinsic synchronicity and entrainment proper is determined. How does the system “know” when to focus more on the bottom-up input (entrainment proper), and when instead to rely more on top-down information (intrinsic synchronicity) to drive the oscillatory behaviour in auditory cortices? If the rate of intrinsic synchronicity is different from that of entrainment proper, would this be expected to result in competition between endogenously generated expectations and exogenous features of the speech input in determining how the speech signal is segmented? These are important questions, because over-reliance on top-down information may lead to inappropriate or incorrect speech segmentation when expectations only partially match the input. The degree to which this occurs (c.f., Bosker, 2017; Kösem et al., 2018), and the severity of the consequences (e.g. for understanding an interlocutor’s intended meaning) in cases where it does, will be important to assess. Indeed, recent proposals suggest that an inappropriate weighting of top-down and bottom-up information may be an important factor contributing to psychosis (Sterzer et al., 2018) and that neural synchronisation may provide a useful “window” for monitoring such (im)balances (e.g. Lakatos, Schroeder, Leitman, & Javitt, 2013). Similarly, if too much weight is given to entrainment proper, one is right back to the starting point that speech is not perfectly rhythmic and so seems unlikely to be the sole driver of strong entrainment in auditory cortices. Some relevant factors in determining this weighting might be things like the likelihood that the input signal is speech (or speech-like), how reliable the speech signal is (in a noisy environment, for instance, one might expect a higher weighting of intrinsic synchronicity), the amount of relevant top-down information that has been accumulated, and possibly the estimated degree of variability in the syllable or phoneme rates.

Some speculative consequences

If MSM turn out to be correct that much of what we call entrainment is actually internal synchronicity in disguise, one conclusion we might draw is that the only speech-specific entrainment is entrainment proper. Put another way, internal synchronicity at rates corresponding to inferences about abstract linguistic information, that does not (necessarily) express a discrete physical signature in the speech signal (e.g. morphemes, lexical representations, syntactic and semantic structures, discourse and event structures), might be expected to exhibit similar synchronicity for input modalities other than speech; like in reading for example. This possibility that entrainment may be observed during reading and not just listening to speech is intriguing and warrants further experimental investigation. In fact, there is already quite a bit of evidence for internal synchronicity during reading (for a recent review see Prystauka & Lewis, 2019). What has not been observed is entrainment proper related to visual sampling while reading, akin to the entrainment proper observed while listening to speech. One potential reason for this is that most studies of reading using M/EEG employ a rapid serial visual presentation (RSVP) approach for presenting words, as this typically makes subsequent data analyses less complex and more systematic. It is possible, however, that this approach obscures any entrainment proper that may be observable when reading naturally (i.e. executing saccadic eye movements to scan the page in the service of information uptake; Rayner, 1998). As one might expect, there are systematic differences between the eye movement behaviour of fast and slow readers (Hawelka, Schuster, Gagl, & Hutzler, 2015). Artificially imposing a particular reading rate as is typically done with RSVP may interfere with or reduce any entrainment proper that might arise when reading naturally. Fixation-related approaches to the analysis of M/EEG data recorded simultaneously with people’s eye movements during naturalistic text reading (e.g. Dimigen, Sommer, Hohlfeld, Jacobs, & Kliegl, 2011; Himmelstoss, Schuster, Hutzler, Moran, & Hawelka, 2019; Metzner, von der Malsburg, Vasishth, & Rösler, 2017) offers the potential to further probe whether such entrainment proper can be observed during naturalistic reading.

One fascinating aspect of this question will be whether entrainment-like behaviour is observed in auditory cortices, or instead in the visual system (or perhaps both). This would speak directly to models of reading (e.g. Clifton et al., 2016; Reichle, Rayner, & Pollatsek, 2003; Seidenberg & Plaut, 1998) and questions about the degree to which grapheme-to-phoneme mapping may be mediated by neural synchronicity. There is already evidence that lip-reading and co-speech gestures (both involving input from the visual modality) accompanying speech input can lead to (increased) entrainment in auditory and visual regions (Brookshire, Lu, Nusbaum, Goldin-meadow, & Casasanto, 2017; Crosse, Butler, & Lalor, 2015; O’Sullivan, Crosse, Di Liberto, & Lalor, 2017). On the other hand, visual sampling is known to be (quasi-)rhythmic in nature (e.g. VanRullen, Zoefel, & Ilhan, 2014), and is crucially mediated by attention. This suggests that we might (also) expect to observe entrainment proper in the visual system during reading. Just like for speech, it will be important investigate the extent to which this entrainment proper (assuming that it is observed for reading) is mediated by more abstract linguistic information through internal synchronicity.

Acknowledgements

Many thanks to Julie Van Dyke for our endless discussions about potential relationships between eye movements during reading and neural entrainment.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

A.G.L. was supported by Gravitation Grant 024.001.006 of the Language in Interaction Consortium from Netherlands Organisation for Scientific Research.