Neural network processing of natural language: I. Sensitivity to serial, temporal and abstract structure of language in the infant

Abstract

Well before their first birthday, babies can acquire knowledge of serial order relations (Saffran et al., 1996a), as well as knowledge of more abstract rule-based structural relations (Marcus et al., 1999) between neighbouring speech sounds within 2 minutes of exposure. These early learners can likewise acquire knowledge of rhythmic or temporal structure of a new language within 5-10 minutes of exposure (Nazzi et al., 1998). All three of these types of knowledge likely play invaluable roles in "bootstrapping" language acquisition. Two important open questions that remain include: What are the mechanisms that provide this rapid learning ability, and how do they depend on pre-exposure to the environment? Here we show that a neurophysiologically validated temporal recurrent network simulates babies' capabilities to learn serial order and rhythmic structure. Indeed the recurrent network is capable of representing serial and temporal structure with no pre-exposure, and through exposure these internal representations can become bound to behavioural responses. In contrast, babies' performance in extracting abstract structure can only be simulated by a modified version of the model. We thus demonstrate how innate representational capabilities for serial and temporal structure of language could arise from a common neural architecture, distinct from that required for the representation of abstract structure, and we provide a predictive testable model of at least these aspects of the initial computational state of the language learner.