Evidence for goodness of fit in Karl Pearson chi-squared statistics

ABSTRACT

Chi-squared tests for lack of fit are traditionally employed to find evidence against a hypothesized model, with the model accepted if the Karl Pearson statistic comparing observed and expected numbers of observations falling within cells is not significantly large. However, if one really wants evidence for goodness of fit, it is better to adopt an equivalence testing approach in which small values of the chi-squared statistic indicate evidence for the desired model. This method requires one to define what is meant by equivalence to the desired model, and guidelines are proposed. It is shown that the evidence for equivalence can distinguish between normal and nearby models, as well between the Poisson and over-dispersed models. Applications to the evaluation of random number generators and to uniformity of the digits of pi are included. Sample sizes required to obtain a desired expected evidence for goodness of fit are also provided.

Keywords:

• Effect size
• equivalence testing
• Kullback–Leibler divergence
• over-dispersion
• variance stabilizing transformation

Acknowledgments

The author thanks Natalie Karavarsamis for helpful comments on an early version of this manuscript and to Yuri Nikolayevsky, Robin Hill and Luke Prendergast for discussions and suggestions on a later draft. In addition, the author is indebted to the referee and Editors for their challenging and probing questions which improved the content and presentation of the text.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

Notes: The sup metric distance M and symmetrized Kullback–Leibler divergence J of $\mathit{p}$ from $\mathit{u}$ is tabled for each model. Also listed is the estimated power ${\mathrm{\Pi }}_{0.05}({\mathit{p}}_j)=P_{{\mathit{p}}_j}\{S\ge c\}$, where $c={\chi }_5^2(0.95)$ of the level 0.05 ${\chi }^2$ test for non-uniformity.

Notes: The sample sizes are for the maximum relative error k = 1; for smaller k multiply these entries by $1/k^2$.

Notes: For each sample the evidence for normality was computed using the method of Section 4.1 with r given by (13(13) $r=max\{10,\lceil \ln (n)\rceil \}.$(13) ) and the mean and standard deviation $\overline{T}(s_T)$ were recorded and listed in the last three columns above. The maximum expected evidence when the data are normal is $m_0=\sqrt{{\lambda }_0+\nu /2}-\sqrt{\nu /2}$, where $\nu =r-3$ and ${\lambda }_0=n/\{4(r-1)\}$.

Note: The means(standard deviations) of the replicated values are listed.