Evaluation measures and the study of language acquisition

Language acquisition has traditionally provided one of the primary explanatory goals of theory construction in linguistics (Chomsky 1965). A theory of grammar aims not merely to provide analyses for particular phenomena in particular languages, but rather to characterize the nature of possible human languages. In turn, constraints on the range of possible languages should make it possible for learners to generalize from their finite experience to a grammatical system capable of generating an infinite number of sentences, and to do so in a roughly uniform way that is robust to small variations in experience.

In light of this explanatory agenda, research in developmental linguistics has two goals. First, it aims to test a theory of linguistic universals and cross-linguistic variation by showing how that space constrains the hypotheses that learners consider during development (Pinker 1984, Gleitman 1990, deVillers, Roeper & Vainikka 1990, Thornton 1990, Crain 1991, Snyder 2001, Lidz, Waxman & Freedman 2003, Syrett & Lidz 2009, Bergelson & Idsardi 2009, Berent 2013). Do learners only consider hypotheses within the space of possible grammars defined by a linguistic theory? To the extent that they consider hypotheses outside this space, this provides an argument against some piece of the linguistic theory. Second, developmental linguistics aims to understand how learners use the input to traverse the space of possible languages (Wexler & Culicover 1980, Dresher & Kaye 1990, Lightfoot 1991, Gibson & Wexler 1994, Tesar & Smolensky 1998, Pater 2008, Yang 2004, Viau & Lidz 2011, Sakas & Fodor 2012, Pearl & Sprouse 2013). Do learners use all of the data that they are exposed to in determining the grammar? How are grammars updated throughout development? Is grammar acquisition deterministic?

While generative research in first and second language acquisition uses the descriptive vocabulary of theoretical linguistics to understand language development, fewer and fewer papers address these fundamental questions and link a theory of possible grammars to observations about linguistic development. In large measure, this is for good reason. Often, psycholinguistic factors conspire to give the illusion that learners have acquired a grammar unlike that of their environment or outside the space of possible grammars. Thus, we find a focus on diagnosing learners’ patterns of behavior as being due to grammatical knowledge or to factors of language use (Hamburger & Crain 1984, Crain & Thornton 1998, Musolino & Lidz 2006, Conroy et al 2009, Clahsen & Felser 2006). Indeed, the study of the extralinguistic contributions to both first and second language acquisition has become a growing subfield of its own (Omaki & Lidz, 2015. Phillips & Ehrenhofer 2015). Relatedly, identifying the grammatical knowledge of children and ruling out alternative grammatical analyses is, in itself, a daunting task for any given piece of knowledge (Valian 1991, Hoekstra & Hyams 1998, Thornton & Wexler 1999, Becker 2005, Valian, Solt & Stewart 2009, Orfitelli & Hyams 2012 inter alia). As a consequence, the field is largely dominated by investigations of children’s knowledge at specific ages, the trajectory of linguistic development and the interaction between knowledge and performance factors in shaping children’s linguistic behavior and development. However, the subsequent step of using those results to inform questions of linguistic theory or to inform the theory of how learners use their experience to move through the space of possible grammars is rarely undertaken (but see Yang 2004, Becker & Mitchener 2011, Pearl 2011 as prominent counterexamples).

Given that 2015 was the 50th anniversary of Aspects of the Theory of Syntax (Chomsky 1965), a book that articulated the explanatory goals of generative linguistics, a specially themed GALANA was held in 2015 at the University of Maryland in order to highlight research that foregrounds these traditional questions. The conference theme addressed the question of how linguistic theory informs our understanding of both what children know at any given point in development, and also how the input can allow for the construction of a particular grammar.

Chomsky (1965) proposed that a language acquisition mechanism must minimally consist of three components: a way of representing the input signal, a space of possible grammars and an evaluation measure—a method for selecting a particular grammar on the basis of the first two components. However, even as grammatical theories have diversified from sets of rules (Chomsky 1965) to principles and parameters (Chomsky 1981, Dresher & Kaye 1990, Baker 1995) or constraint rankings (Prince & Smolensky 1993), the problem of grammatical opacity has remained. The utterances that a learner is exposed to bear an indirect relation to the grammar that generated them, even with highly constrained grammatical theories. The vast majority of surface evidence is compatible with multiple grammatical descriptions (Berwick 1985, Clark 1989, 1992, Gibson & Wexler 1994, Dresher & Kaye 1990, Tesar & Smolensky 1998, Yang 2004, Pearl 2009, Fodor & Sakas 2013). Consequently, even defining the evidence that unambiguously allows for the selection of one particular grammar remains a challenge, let alone identifying the psychological mechanisms through which learners recognize that evidence and put it to use in grammar selection (Lidz & Gagliardi, 2015).

The current special issue presents four articles by leading researchers in the field, all of whom use computational tools to address the opacity of the relation between a grammar and the set of utterances it produces. In this light, the evaluation measure is the mechanism through which this opacity can be overcome.

Janet Dean Fodor’s article approaches the problem of parametric ambiguity. In a theory of grammatical variation defined by syntactic parameters, the evidence a learner is exposed to may be compatible with multiple grammars. Fodor proposes an evaluation measure that inherits its properties from sentence processing mechanisms. She argues that an incremental parser with limited capacity for parallelism circumvents certain problems of parametric ambiguity. In this sense, the evaluation measure is a byproduct of the parser, rather than a component of the learning theory per se. She argues that parameters should be thought of as tree fragments that can be activated with a strength proportional to their frequency. With this conception of the parameters, she reports data that support a mechanism of learning by parsing over one that learns through domain search.

Charles Yang addresses the puzzle of generalization and whether indirect negative evidence is a viable information source for learning morphosyntactic generalizations. He focuses in particular on subset-superset problems (Gold 1967, Berwick 1985) in which a given set of data is compatible with two grammars whose extensions stand in a subset-superset relation. In examining the subset of adjectives in English that can be used predicatively (I am asleep) but not attributively (*the asleep cat), Yang shows that learning by indirect negative evidence fails to identify this subclass accurately and that instead, a grammatical analysis that treats them as belonging to the same syntactic class as prepositional phrases can lead to successful learning on the basis of positive evidence alone. Yang also identifies a common threshold against which any set of forms can be compared in order to determine whether there is a productive generalization that explains those forms. In essence, Yang argues that a learner armed with the right tools of grammatical analysis and a general theory of productivity allows for richer explanations of how learners identify grammatical generalizations than one which simply calculates probabilities. More generally, Yang’s article emphasizes that learners require a rich characterization of possible grammatical representations in order for simple evaluation measures to be effective.

Lisa Pearl raises the question of whether one can use evaluation measures as a probe of grammatical representations. She compares several models of stress placement, which differ in their representational vocabulary, but which all provide accurate descriptions of English stress placement. She asks within each model whether a domain-general bayesian learner converges on the grammar of English stress given child directed speech. Then, she compares differences in convergence in order to see whether one class of models gets closer to the correct grammar. If such a way of evaluating grammars works, it opens up new ways of teasing apart alternative grammatical analyses. That is, to the extent that two grammars with equal descriptive coverage differ in how well they can be learned, we will have used the evaluation measure as a way of separating descriptive from explanatory adequacy.

Finally, Bruce Tesar examines learnability in the context of Optimality Theory (Prince & Smolensky 1993). The puzzle in this context comes from the vast space of possible grammars and lexicons defined by an OT framework. Searching this vast space is computationally infeasible in both humans and machines. Tesar’s approach to delimiting this search space takes advantage of output driven maps. Such maps define candidates in terms of the similarities between the output form and the potential input forms. Because entailment relations can be defined between large swaths of candidate input-output maps, the search space for a learner can be defined in highly structured ways that allow for more efficient search over possible lexicons. Tesar illustrates how such an approach can be taken to learning phonotactics, morphophonological generalizations, underlying features, constraint rankings, and contrastiveness. These output driven maps define structure over possible grammars beyond what comes from the OT framework itself, making more efficient learning possible.

While the four articles all tackle different problems in syntax, morphology and phonology, they share an aim of exploring how learners search through the space of possible grammars to identify a particular grammar. In each case we see how attention to the fine structure of grammatical theory along with computational and psycholinguistic considerations contributes to new insights in the study of language acquisition. These articles thus provide a model for how future research in developmental linguistics can reconnect with the fundamental explanatory goals of linguistic theory.

Acknowledgements

The conference that gave rise to the papers in this special issue was funded in part by a grant from the National Science Foundation: BCS-1451584.