The role of corpus size and syntax in deriving lexico-semantic representations for a wide range of concepts

Abstract

One of the most significant recent advances in the study of semantic processing is the advent of models based on text and other corpora. In this study, we address what impact both the quantitative and qualitative properties of corpora have on mental representations derived from them. More precisely, we evaluate models with different linguistic and mental constraints on their ability to predict semantic relatedness between items from a vast range of domains and categories. We find that a model based on syntactic dependency relations captures significantly less of the variability for all kinds of words, regardless of the semantic relation between them or their abstractness. The largest difference was found for concrete nouns, which are commonly used to assess semantic processing. For both models we find that limited amounts of data suffice in order to obtain reliable predictions. Together, these findings suggest new constraints for the construction of mental models from corpora, both in terms of the corpus size and in terms of the linguistic properties that contribute to mental representations.

Keywords:

• Semantic memory
• Text corpora
• Syntactic dependency
• Word associations
• Similarity

Notes

¹The study is ongoing at http://www.smallworldofwords.com/nl and currently contains data for over 16,000 cue words.

²In past studies we have applied -score weighting, as this consistently improved the estimates of the word-association measures over a range of tasks. In this study, PMI was nevertheless chosen to increase the comparability of the text-model where PMI is applied as standard.

³The 95% confidence intervals used here encompass a significance test but also provide an estimate of the magnitude of the effect. If zero is included in the confidence interval, the result will not reach the 5% significance level.