Word Use Equivalence and Hierarchical Word Tiers

ABSTRACT

A ranked word list provides information about the position of each word in the list. However, retaining and employing the measure used to generate the ranked list can yield additional information about the words. If $\omega$ denotes the prevalence of a word in a corpus, then not only can the values of $\omega$ be ordered, their values can be compared to one another, and words having similar values can be grouped together into equivalence classes. Measures of word prevalence include mean text frequency, the dispersion of words across texts in a corpus, or a measure that combines frequency and dispersion. In this paper, we examine the concepts of word equivalence classes and hierarchical word tiers and apply these concepts to the words in the British National Corpus (BNC). Hierarchical word tiers can be constructed without the knowledge of all pairwise comparisons of the words under study. By grouping words that have similar values of prevalence, the ranked ordered list reduces to an informative set of hierarchical word tiers where each tier contains words that are similar to one another in terms of their use in the corpus.

Acknowledgments

The authors thank the two anonymous reviewers for their valuable comments that substantially improved the quality of this paper.

Disclosure Statement

No potential conflict of interest was reported by the author(s).