Unifying Models for Word Length Distributions Based on Types and Tokens

ABSTRACT

Word length studie^s have been one of the central issues in Quantitative Linguistics for a long time. Most models were constructed for very specific purposes, i.e. the individual models apply only to a specific language, only to token counts or only to type counts. The present paper takes up the challenge of developing unifying models which account for both type and token frequencies of a moderately large sample of languages (eight Indo-European and two non-Indo-European languages). We introduce three models which can be well fitted to all our data: the exponentiated Hyper-Poisson distribution, the generalized gamma and the Sichel distribution. We also discuss the possibility of interpreting the model parameters linguistically.

Acknowledgments

The authors would like to thank an anonymous referee for useful comments on earlier versions of the manuscript which helped us to improve the paper considerably.

Disclosure statement

No potential conflict of interest was reported by the authors.