Introduction to the special issue emergence of speech and language from prediction error: error-driven language models

ABSTRACT

Last year, 2022, marked the 50th anniversary of the Rescorla-Wagner learning equations – a landmark in the development of learning theory. Originally based on animal learning, the equations and error-driven learning models as a whole were rapidly adopted and applied in several areas of human psychology. While language acquisition research initially took a different path, interest in the role of error-driven learning in language is growing. With the aim of strengthening this emerging research field, sparking discussion and cross-fertilisation of ideas across linguistics, psychology and cognate fields, this Special Issue presents nine papers that address the role of error-driven learning in language. The papers investigate a wide range of subfields of linguistics, and include two review articles, in addition to computational modelling and experimental research. The collection thus serves both as an introduction for those new to the subject and as an overview for those already working in the field.

KEYWORDS:

• Error-driven learning
• discriminative learning
• language acquisition
• language processing

1. Introduction

This is a critical moment in the field of language research. The last several decades of empirical research in linguistics have witnessed a dramatic shift in the fundamental assumptions researchers hold about language. The previously dominant idea that language was largely innate and relied on specialised biological systems (Chomsky & Halle, 1968; Eimas & Corbit, 1973; Eimas et al., 1971), is increasingly giving way to the idea that language emerges from general learning mechanisms (see Frost et al., 2019; Hasson, 2017; Perruchet & Pacton, 2006; Rebuschat & Monaghan, 2019, for reviews; and Nixon & Tomaschek, 2023, for discussion related to speech sounds). We are now faced with a new problem: if language is learned, what cognitive mechanisms are responsible? And what implications does this have for everyday language use? There has also been increasing interest in evolving the field of linguistics by developing robust models of language that draw not only on experimental data, but are also supported by mathematical and computational modelling (Frost et al., 2015; MacWhinney, 2010; McMurray et al., 2009; McMurray & Hollich, 2009; Monaghan & Rowland, 2017).

Meanwhile, in a separate area of psychology, robust theoretical, mathematical and computational models of learning, known as error-driven learning models, have been developed. The models were originally based on animal learning research that could be said to have started with Pavlov, and kicked off in earnest at least as early as the 1960s (Kamin, 1968, 1969; Rescorla, 1968, 1988; Siegel & Allan, 1996; Widrow & Hoff, 1960), eventually leading to the highly influential Rescorla-Wagner learning equations (Rescorla & Wagner, 1972).

Due to an unfortunate lack of interdisciplinary communication, this rich knowledge about learning has, until recently, hardly been known in the language sciences. However, language researchers have begun to draw on the insights of learning theory and use the tools of error-driven learning to investigate human learning in linguistic phenomena (Baayen et al., 2016; Chang et al., 2006; Ellis & Sagarra, 2010; Hills et al., 2010; Nieder et al., 2022; Nixon, 2018, 2020; Nixon & Tomaschek, 2020, 2021; Ramscar, Dye, McCauley, 2013; Ramscar & Yarlett, 2007; Ramscar et al., 2010; Tomaschek et al., 2019; Tomaschek & Ramscar, 2022). The fact that the models are mathematically and computationally implemented allows researchers to make strong predictions about linguistic behaviour and to find common mechanisms for seemingly unrelated phenomena. Error-driven learning models are proving to be remarkably powerful for predicting and explaining a wide range of phenomena in human learning, including language acquisition, comprehension and production. However, wide-scale use of these models has not yet reached the language sciences. This may be at least partly due to a lack of awareness of and familiarity with error-driven learning models in the linguistic, psycholinguistic and language science communities. Therefore, one of the key aims of this special issue is to raise awareness of error-driven learning models among researchers in these disciplines. That said, while the focus of this issue is language, the principles of error-driven learning are applicable across many other areas of cognitive science and psychology, such as category learning, memory, visual processing, music cognition, motor control and social cognition, to name but a few. Examination of these models in the present issue may help spark fruitful development or connections with other such disciplines.

But what is “error” and what is its function in learning? Below, we first briefly introduce error-driven learning and the related literature. We then introduce each of the contributions in this collection.

2. Error-driven learning

According to recent theories of learning such as “discriminative learning” (e.g. Ramscar, Dye, McCauley, 2013; Ramscar et al., 2010), the “free energy” theory (e.g. Friston, 2011; Kaplan & Friston, 2018) and connectionist models (Chang, 2002; Chang et al., 2006, 2000; Elman, 1990), learning is based on prediction and feedback from prediction error. Below, we give a brief introduction to error-driven learning models. For a more in-depth introduction and comparison with some other leading models, the reader is referred to Nixon (2020) and Ramscar, Dye, McCauley (2013).

At any given moment, incoming sensory cues are potentially used to make predictions about upcoming events (outcomes in the error-driven learning parlance;¹ Hoppe et al., 2020; Nixon, 2020; Nixon & Tomaschek, 2020, 2021; Ramscar et al., 2010). In error-driven learning models, these predictions – i.e. expectations – are represented as connections between cues and outcomes; the more expected an outcome is, based on a cue (or set of cues), the greater the connection strength (or connection weights). Learning is implemented as incremental adjustments to these connection weights.

Closely related to the concept of expectation, prediction error is essentially surprise. A common misinterpretation of error is to think of it as equivalent to a “mistake” – along the lines of “I expected X, but Y happened”. Rather, prediction error is a gradient measure of the difference between the degree of expectation of an outcome and the observed occurrence (or non-occurrence) of the outcome. As such, if I have a 90% expectation of X, and X occurs, a 10% prediction error remains (the difference between 90% and 100%); on the other hand, if I have a 90% expectation of X, and X does not occur, I have 90% prediction error (the difference between 90% and 0%).²

This numerical calculation of the degree of expectation and error is crucial to the model. These calculations are the basis for estimating learning. Learning can be estimated incrementally in what are called learning events: for example, each trial of an experiment. The degree of learning in each learning event is proportional to the prediction error in that learning event. Highly expected outcomes, if they occur, yield less prediction error – and thus less learning – than outcomes that are less expected. Conversely, when an expected outcome does not occur, this results in more error – and therefore more learning – than when an unexpected outcome does not occur.

In the Rescorla-Wagner equations (Rescorla & Wagner, 1972), outcomes are considered to have maximum and minimum limits on expectation. Expectation tends to start at or near zero early in learning, and when cues predict the occurrence of an outcome, expectation increases with experience. Crucially, as a result of adjustments being proportional to prediction error, learning is not linear, but rather takes the shape of the famous “learning curve”: learning is faster early on and slows over time as expectation increases.

Another important aspect of the model is that any adjustments made to connection weights in a given learning event are divided and shared equally between all cues present in that learning event.³ This means that cues compete for predicting outcomes. Cue competition has important consequences. One well-known example of how cue competition affects learning is when there are differences in the relative timing of learning different cues. If certain cues are encountered early on in learning (near the beginning of the learning curve), they may gain a bigger share of learning, due to weight adjustments being larger at that early stage. Later, when learning approaches asymptote – i.e. when an outcome is highly expected and uncertainty is low – any new cues introduced at this point are not learned well. This “blocking effect” was first demonstrated in Kamin's animal learning experiments (Kamin, 1969). In an artificial language learning study, Nixon (2020) showed this blocking effect also occurs in speech acquisition and may be an important factor in understanding how second language acquisition differs from first language acquisition.

The principles of error-driven learning are computationally implemented most famously in Rescorla & Wagner's (1972) learning equations (Rescorla & Wagner, 1972), although a similar algorithm was also proposed by Widrow and Hoff (1960, see also Stone, 1986). It is formally equivalent to the delta-rule used in connectionist networks (Sutton & Barto, 1981) and equivalent to learning algorithms in most neural networks (Ng & Jordan, 2002). The Rescorla-Wagner learning equations have been used to predict learning and human performance in the domains of morphological learning in children (Ramscar, Dye, Kelin, 2013; Ramscar, Dye, McCauley, 2013; Ramscar et al., 2011, 2010), morphological processing in adult perception and production (Nieder et al., 2021, 2022; Tomaschek et al., 2019; Tomaschek & Ramscar, 2022), auditory comprehension and word recognition (Arnold et al., 2017; Baayen et al., 2016; Shafaei-Bajestan & Baayen, 2018) and speech acquisition and phonetic learning in adult second language learners (Nixon, 2018, 2020) and infants (Nixon & Tomaschek, 2020, 2021). The papers in this Special Issue add to the growing literature exploring error-driven learning as a mechanism contributing to language acquisition and processing, perception, production and comprehension. The papers are summarised below.

3. This issue

The nine papers in this issue cover a broad range of subfields of linguistics: from phonetic learning (McMurray, 2023), morphology (Ramscar, 2023), semantics (Filipović Ðurđević & Kostić, 2023; Luo et al., 2023) and word recognition (Shafaei-Bajestan et al., 2023) to structural priming (Khoe et al., 2023), letter sequences (Kapatsinski, 2023), grammar acquisition in sentence processing (Bröker & Ramscar, 2023) and memory formation during reading (Haeuser & Kray, 2023). The broad scope of topics illustrates the potential of error-driven learning as a unifying theory of language acquisition and processing. In saying this, we do not assume that error-driven learning is the only learning mechanism involved; however, it appears to be involved at all linguistic levels.

Underlying error-driven learning theory are computational implementations that allow researchers to make precise predictions about human behaviour, neural signals or any other measurable effects of learning. Several papers use R packages such as NDL (Arppe et al., 2018), EDL (van Rij & Hoppe, 2021) or LDL (Baayen et al., 2019) which are based on the Rescorla-Wagner equations (Rescorla & Wagner, 1972). Others use deep learning (Luo et al., 2023) or the Dual-path model proposed by (Chang et al., 2023; Khoe et al., 2023). Two papers in this Special Issue specifically address the computational implementation, either discussing aspects of the algorithm (Kapatsinski, 2023) or the input representations (Bröker & Ramscar, 2023).

3.1. Phonetics

McMurray (2023) provides an extensive review of the state of the art in speech acquisition research. He re-examines the question of how it is that infants learn the phonetic space of their language. In what McMurray terms the “standard model”, it is often assumed that infants learn native language speech sounds very early on, within the first year, and that – since it appears that there are no overt “teaching signals” – this must be achieved via unsupervised learning mechanisms, such as distributional learning. However, McMurray (2023) argues against this view for a number of reasons. In particular, he argues that the idea that there were no teaching signals available was premised on the assumption that teaching signals needed explicit instruction or corrections. However, in an error-driven learning framework, the teaching signals are essentially always available, because they result from the process of prediction and prediction error – even in what McMurray calls an “unsupervised ecology”.

3.2. Morphology

Traditionally, it was assumed that regularly inflected word forms were obtained through the application of rule-based processes, while irregularly inflected word forms were stored in the lexicon (see e.g. Pinker & Prince, 1994). However, Rumelhart and McClelland (1986) presented a computational model that could produce regular and irregular word forms through analogy learned by means of a single learning mechanism. Ramscar (2023) further examines Rumelhart and McClelland (1986)'s proposal. He discusses how the knowledge required to understand and produce regularly and irregularly inflected words is obtained through prediction and prediction-error. A review is presented of how this knowledge is applied to novel words from an error-driven learning perspective, taking into account developmental changes in children's morphological capabilities as well as context-dependent variation in inflection.

3.3. Semantics

Filipović Ðurđević & Kostić (2023) take an information theoretic and discriminative learning approach to understanding polysemy. The term polysemy is usually defined as the characteristic of a word having multiple senses. As such, polysemy is a form of ambiguity. Typically, the degree of ambiguity of polysemous words is considered to depend on the number of senses. However, Filipović Ðurđević & Kostić (2023) refine this measure of ambiguity. Rather than discrete counts of the number of senses, they reinterpret polysemy as a continuous measure of sense diversity or sense uncertainty. They use these information theoretic measures and discriminative learning to examine the question of how polysemy is processed during visual lexical decision.

Luo et al. (2023) use a deep learning model to show how words shape semantic processing through a discriminative process – labels (i.e. words) that are shared by objects or events highlight commonalities between these objects or events, while contrasting labels highlight differences. Differences in the use of labels might arise, for example, from differences between languages or differences in levels of expertise. For example, do all dogs share the same label or are dogs labelled by their breed? Luo et al. (2023) show how these differences in label use lead to differences in the semantic representations underlying the labels; in other words, they demonstrate a means by which language can shape thought.

3.4. Word recognition

Shafaei-Bajestan et al. (2023) use a computational modelling experiment to address the role of discrete units in word recognition. Although most theoretical and computational work proposes that discrete phones function as an interface between the variable acoustic information listeners perceive and meaning, there has been strong interest in developing models that do not assume discrete units (see Nixon & Tomaschek, 2020, 2021 for error-driven learning models that do not assume discrete units and Nixon & Tomaschek, 2023 for a review). To this end, Shafaei-Bajestan et al. (2023) modelled word recognition without any discrete units, either in the acoustic input or in the semantic output. Instead, they used a gradient representation of meaning and acoustic input obtained from spontaneous speech of many speakers to train their model. On the basis of their results, they argue that word recognition does not need an interface in the form of discrete units. Instead, observed discrete units are actually an epiphenomenon of phone-like clustering of acoustic cues due to learning.

3.5. Sentence processing

In three simulations of bilingual sentence production, Khoe et al. (2023) test whether cross-linguistic structural priming can be explained by error-driven implicit learning. Previous work has investigated whether participants' choice between two alternative constructions is affected by recent exposure. For example, the dative alternation has two alternative constructions, e.g.: “[subject] showed [direct object] to [indirect object]” or “[subject] showed [indirect object] [direct object]”. Previous studies have shown that when two alternative structures are available, the likelihood of participants selecting a particular alternative is increased by recently reading (or hearing) that alternative – a phenomenon known as structural priming (Bock, 1986; Branigan & Pickering, 2017). This phenomenon has traditionally been understood as resulting from increased activation⁴ of the recently encountered alternative. On the other hand, the Dual-path model of sentence production (Chang, 2002; Chang et al., 2006), proposes that this structural priming effect can instead be explained as a learning effect, rather than an effect of activation in this sense.

Khoe et al. (2023) ran three bilingual models of sentence production (Spanish-English, verb-final Dutch-English, and verb-medial Dutch-English). The simulations were able to capture key findings from behavioural studies of structural priming, supporting the proposal that cross-linguistic structural priming occurs as a result of error-driven implicit learning.

A related question is whether error-driven implicit learning also plays a role in how memory traces are formed while reading. Some studies report that in sentence reading, unpredictable words are better recalled than predictable words, arguing for a “boosting effect” of prediction error in the case of unpredictability. Other studies demonstrate the opposite finding – an effect that is supposed to result from a stronger entrenchment of predictable words in the lexicon. To solve this contradiction, Haeuser & Kray (2023) further investigate the effect of prediction error in a self-paced reading task. They demonstrate that unpredictable words were indeed better recalled. Their result not only supports the idea that words with larger prediction error are better stored in memory, but demonstrates the importance of prediction error for long-term learning.

3.6. Computational implementation

One prediction of the Rescorla-Wagner model is that under certain specific circumstances, due to cue competition, a cue that does not co-occur with an outcome will develop positive connection weights with that outcome. This counter-intuitive prediction has been termed “spurious excitement”. Kapatsinski (2023) puts this prediction to the test with the learning of monosyllabic consonant-vowel words – which occurred either in isolation or in sequences of two or three – as cues to press a left or right arrow. The cues were carefully selected to produce the conditions under which the Rescorla-Wagner model predicts spurious excitement. The experimental results showed no evidence of spurious excitement, calling into question this prediction of the Rescorla-Wagner model. Based on these results Kapatsinski (2023) proposes an adjustment to the Rescorla-Wagner learning equations: rather than a linear activation function, a logistic activation function is proposed, which avoids spurious excitement and yet is still able to capture many of the known learning effects.

Bröker & Ramscar (2023) take an overarching look at the question of what is involved in cognitive modelling and the role of computational modelling in this. They argue that an extremely important – but frequently overlooked – aspect of computational cognitive modelling lies in determining the appropriate input representations to the model. That is, not only is it important to determine the algorithm that best captures (human language) learning, but it is also important to determine what constitutes the input to that learning. To address this, they revisit the question of the role that implicit “negative evidence” plays in language learning. They find that with a certain set of input representations to the model, it is possible to model the results with a dual-mechanism approach, in which children are assumed to switch their assumptions about how the data are sampled depending on the context and adjust their learning strategy accordingly (i.e. between a generative model and a discriminative model). On the other hand, they also find that a single mechanism (discriminative model) approach is possible with a different set of model representations. They conclude that an increased focus on selecting model input representations that faithfully capture the task design could benefit the conceptualisation, evaluation and comparison of cognitive models.

4. Conclusions and future directions

The contributions in this special issue show a wide range of phenomena predicted by error-driven learning models, from behavioural tasks to neural processes, from phonetics to sentence processing. The papers used various different implementations of error-driven learning models, discriminative learning, deep learning and connectionist models, which are all united in their assumption that learning occurs through a process of prediction and feedback from prediction error.

In the transition from animal models to models of human learning, there have been a number of challenges, some of which have been discussed widely in the literature. For example, what is the role of attention in error-driven learning? Should error-driven learning be considered a mechanism for explicit processes or is it purely an implicit learning model? Are there differences between reinforcement learning – i.e. learning involving rewards – and sensory error-driven learning? If so, which best characterises language learning? These will be important questions to investigate in the further development of a learning theory of language.

Acknowledgments

We would like to express thanks for the wonderful editorial support throughout the process of putting together this special issue. Thanks also to the authors for their valuable contributions and hard work, without which this special issue would not have been possible.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This publication was supported by a collaborative grant from the Deutsche Forschungsgemeinschaft (German Research Foundation; Research Unit FOR2373, Project BA 3080/3-2).

Notes

1 In the following, we use the terminology of discriminative learning. Although there are differences in the implementation, the general principles described below apply to all the models.

2 This calculation assumes equivalent calculation of the prediction error for the occurrence vs. non-occurrence of outcomes, i.e. “positive” vs. “negative” evidence. For further discussion of this issue, see Nixon (2020); Ramscar, Dye, McCauley (2013).

3 Some researchers have argued that absent cues may also affect learning (e.g. Van Hamme & Wasserman, 1994), but see Nixon et al. (2022) for discussion and an alternative view.

4 Note that the term “activation” is used in a different sense here to the way it is commonly used in relation to the Rescorla-Wagner equations and similar learning algorithms: in the Rescorla-Wagner equations, activation refers to the sum of connection weights; here it is used in a more general sense commonly used in psycholinguistics to refer to transient neural or cognitive activity related to the item.