Fitting Ranked English and Spanish Letter Frequency Distribution in US and Mexican Presidential Speeches

Abstract

The limited range in the abscissa of ranked letter frequency distributions causes multiple functions to fit the observed distribution reasonably well. In order to critically compare various functions, we apply the statistical model selections on ten functions, using the texts of US and Mexican presidential speeches of the last few centuries. Despite minor switching of ranking order of certain letters during the temporal evolution for both datasets, the letter usage is generally stable. The best fitting function, judged by either least-square-error or by AIC/BIC model selection, is the Cocho/Beta function. We also use a novel method to discover clusters of letters by their observed-over-expected frequency ratios.

Acknowledgements

We would like to thank Osman Tuna Gökgöz for introducing us to the work by Al-Kindi. This work was partially supported by UNAM-PAPIIT project IN115908. The authors wish to thank the hospitality of the Centro de Investigación en Matemáticas Aplicadas, Pachuca, México, where the draft was finalized.