Constraint satisfaction in large language models

ABSTRACT

Constraint satisfaction theories were prominent in the late 20th century and emphasized continuous, rich interaction between many sources of information in a linguistic signal unfolding over time. A major challenge was rigorously capturing these highly interactive comprehension processes and yielding explicit predictions, because the important constraints were numerous and changed in prominence from one context to the next. Connectionist models were conceptually well-suited to this, but researchers had insufficient computing power and lacked sufficiently large corpora to bring these models to bear. These limitations no longer hold, and large language models (LLMs) offer an opportunity to test constraint satisfaction ideas about human language comprehension. We consider how LLMs can be applied to study interactive processes with lexical ambiguity resolution as a test case. We argue that further study of LLMs can advance theories of constraint satisfaction, though gaps remain in our understanding of how people and LLMs combine linguistic information.

KEYWORDS:

• Language comprehension
• constraint satisfaction
• ambiguity
• connectionism
• large language models

Acknowledgements

This work was supported by NSF award 1849236 to the University of Wisconsin–Madison, M.C. MacDonald, PI. We thank Robert Hawkins, Katrin Erk, Loïc Grobol, and two anonymous reviewers for useful comments on an earlier version of this work.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the National Science Foundation: [Grant Number 1849236].