An extension of Monte Carlo hypothesis tests

ABSTRACT

There are many hypothesis testing settings in which one can calculate a “reasonable” test statistic, but in which the null distribution of the statistic is unknown or completely intractable. Fortunately, in many such situations, it is possible to simulate values of the test statistic under the null hypothesis, in which case one can conduct a Monte Carlo test. A difficulty however arises in that Monte Carlo tests, as they are currently structured, are applicable only if ties cannot occur among the values of the test statistics. There is a frequently occurring scenario in which there are lots of ties, namely that in which the null distribution of the test statistic has a (single) point mass. It turns out that one can modify the current form of Monte Carlo tests so as to accommodate such settings. Developing this modification leads to an intriguing identity involving the binomial probability function and its derivatives. In this article, we will briefly explain the modified procedure, discuss simulation studies which demonstrate its efficacy, and provide a proof of the identity referred to above.

KEYWORDS:

• Barnard’s test
• Mixtures
• Monotonicity
• Point mass
• Power

Acknowledgments

This article is based in part on a Summer Research Project undertaken by Celeste Jeffs, under the supervision of Rolf Turner, at the University of Auckland in the summer of 2011/2012. The authors express their warm gratitude to Alastair Scott and to Rachel Fewster of the Department of Statistics, University of Auckland, for some useful advice and suggestions. The authors also thank an anonymous referee whose comments led to improvements in the article.