An extension of Monte Carlo hypothesis tests

There are many hypothesis testing settings in which one can calculate a “reasonable” test statistic, but in which the null distribution of the statistic is unknown or completely intractable. Fortunately, in many such situations, it is possible to simulate values of the test statistic under the null hypothesis, in which case one can conduct a Monte Carlo test. A difficulty however arises in that Monte Carlo tests, as they are currently structured, are applicable only if ties cannot occur among the values of the test statistics. There is a frequently occurring scenario in which there are lots of ties, namely that in which the null distribution of the test statistic has a (single) point mass. It turns out that one can modify the current form of Monte Carlo tests so as to accommodate such settings. Developing this modification leads to an intriguing identity involving the binomial probability function and its derivatives. In this article, we will briefly explain the modified procedure, discuss simulation studies which demonstrate its efficacy, and provide a proof of the identity referred to above.