Why Do Parameter Values in the Zipf-Mandelbrot Distribution Sometimes Explode?

Figures & data

Table 1. Grapheme rank-frequency distributions in Bulgarian and Slovene (data – observed frequencies, ZM – expected frequencies from the right truncated ZM distribution, geom – expected frequencies from the right truncated geometric distribution).

	Bulgarian			Slovene
rank	data	ZM	geom	data	ZM	geom
1	36,841	31,068.29	30,967.82	30,849	31,854.82	31,789.57
2	24,724	27,660.13	27,605.25	29,708	28,555.61	28,524.02
3	23,098	24,629.06	24,607.80	26,129	25,601.07	25,593.92
4	21,644	21,932.99	21,935.82	25,886	22,954.86	22,964.82
5	19,535	19,534.59	19,553.97	17,175	20,584.55	20,605.78
6	17,133	17,400.72	17,430.75	15,921	18,461.12	18,489.08
7	13,867	15,501.95	15,538.07	15,045	16,558.63	16,589.81
8	13,394	13,812.15	13,850.91	14,144	14,853.90	14,885.64
9	12,224	12,308.14	12,346.94	14,139	13,326.20	13,356.53
10	11,329	10,969.30	11,006.28	12,402	11,956.98	11,984.49
11	9197	9777.36	9811.19	11,569	10,729.66	10,753.40
12	8542	8716.05	8745.86	11,412	9629.42	9648.77
13	7950	7770.93	7796.21	10,029	8642.97	8657.61
14	7339	6929.18	6949.68	9167	7758.46	7768.27
15	6197	6179.39	6195.07	8753	6965.25	6970.28
16	5633	5511.44	5522.39	6441	6253.84	6254.26
17	5309	4916.31	4922.75	5515	5615.72	5611.80
18	4770	4385.99	4388.23	5336	5043.28	5035.34
19	4554	3913.37	3911.74	4755	4529.70	4518.09
20	4344	3492.12	3487.00	4429	4068.87	4053.97
21	4035	3116.61	3108.37	3054	3655.33	3637.53
22	3220	2781.82	2770.85	2923	3284.19	3263.87
23	2681	2483.30	2469.99	1967	2951.05	2928.59
24	2197	2217.10	2201.79	1893	2652.01	2627.76
25	1956	1979.68	1962.71	230	2383.53	2357.82
26	1936	1767.90	1749.60
27	1464	1578.97	1559.62
28	362	1410.41	1390.27
29	336	1259.99	1239.31
30	320	1125.76	1104.75
		a = 103.32	p = 0.1086		a = 103.24	p = 0.1027
		b = 887.7			b = 942.74
		C = 0.0165	C = 0.0164		C = 0.0195	C = 0.0193

Table 2. Convergence of the ZM distribution to the geometric distribution (max – the maximum number of iterations in the function optim; a, b – optimized parameters of the ZM distribution).

Bulgarian					Slovene
max	a	b	$1-e^{-\frac{a}{b}}$	C	max	a	b	$1-e^{-\frac{a}{b}}$	C
70	16.08	125.87	0.1199	0.0170	20	11.69	97.36	0.1131	0.0215
75	32.88	271.22	0.1142	0.0166	30	14.26	122.23	0.1101	0.0211
80	51.03	433.13	0.1111	0.0165	40	57.17	512.03	0.1056	0.0198
90	89.47	766.78	0.1101	0.0165	50	101.58	928.80	0.1036	0.0196

Figure 1. Rank-frequency distribution of graphemes in Slovene (white – observed frequencies, grey – expected frequencies from the right truncated ZM distribution, black – expected frequencies from the right truncated geometric distribution).