A method for constructing exact tests from test statistics that have unknown null distributions

A randomized procedure is described for constructing an exact test from a test statistic F for which the null distribution is unknown. The procedure is restricted to cases where F is a function of a random element U that has a known distribution under the null hypothesis. The power of the exact randomized test is shown to be greater in some cases than the power of the exact nonrandomized test that could be constructed if the null distribution of Fwere known.     

Testing the equality of several covariance matrices

Using a new approach based on Meijer G-functions and computer simulation, we numerically compute the exact null distribution of the modified-likelihood ratio statistic used to test the hypothesis that several covariances matrices of normal distributions are equal. Small samples of different sizes are considered, and for the case of two matrices, we introduce a new test based on determinants, with the null distribution of its criterion also fully computable. Comparisons with published results show the accuracy of our approach, which is proved to be more flexible and adaptable to different cases.     

Range-preserving unbiased estimators in the poisson case

Abstract

The Kruskal–Wallis test is a popular nonparametric test for comparing k independent samples. In this article we propose a new algorithm to compute the exact null distribution of the Kruskal–Wallis test. Generating the exact null distribution of the Kruskal–Wallis test is needed to compare several approximation methods. The 5% cut-off points of the exact null distribution which StatXact cannot produce are obtained as by-products. We also investigate graphically a reason that the exact and approximate distributions differ, and hope that it will be a useful tutorial tool to teach about the Kruskal–Wallis test in undergraduate course.     

A Permutation Test for Planar Regression

Oja (1987) presents some distribution-free tests applicable in the presence of covariates when treatment values are randomly assigned. The formulas and calculations are cumbersome, however, and implementation of the tests relies on using a x² approximation to the exact null distribution. In this paper a re-formulation of his test statistic is given which has the advantages of ease of calculation, explicit formulas for permutation moments, and allowing a Beta distribution to be fitted to the exact null distribution.     

Approximate and exact distributions of rank tests for balanced incomplete block designs

Judges rank k out of t objects according to m replic ations of abasic balanced incomplete block design with bblocks. In Alvo and Cabilio(1991),it is shown that the Durbin test, which is the usual test in this situation, can be written in terms of Spearman correlations between the blocks, and using a Kendall correlation, they generated a new statistic for this situation.This Kendall tau based statistic has a richer support than the Durbin statistic, and is at least as efficient.In the present paper,exact and simulation based tables are generated for both statistics, and various approximations to these null distributions are considered and compared.     

A new ranked set sampling protocol for the signed rank test

In this paper, we construct a new ranked set sampling protocol that maximizes the Pitman asymptotic efficiency of the signed rank test. The new sampling design is a function of the set size and independent order statistics. If the set size is odd and the underlying distribution is symmetric and unimodal, then the new sampling protocol quantifies only the middle observation. On the other hand, if the set size is even, the new sampling design quantifies the two middle observations. This data collection procedure for use in the signed rank test outperforms the data collection procedure in the standard ranked set sample. We show that the exact null distribution of the signed rank statistic W_RSS⁺ based on a data set generated by the new ranked set sample design for odd set sizes is the same as the null distribution of the simple random sample signed rank statistic W_SRS⁺ based on the same number of measured observations. For even set sizes, the exact null distribution of W_RSS⁺ is simulated.     

Negative exponential disparity based family of goodness-of-fit tests for multinomial models

This paper investigates a new family of goodness-of-fit tests based on the negative exponential disparities. This family includes the popular Pearson's chi-square as a member and is a subclass of the general class of disparity tests (Basu and Sarkar, 1994) which also contains the family of power divergence statistics. Pitman efficiency and finite sample power comparisons between different members of this new family are made. Three asymptotic approximations of the exact null distributions of the negative exponential disparity famiiy of tests are discussed. Some numerical results on the small sample perfomance of this family of tests are presented for the symmetric null hypothesis. It is shown that the negative exponential disparity famiiy, Like the power divergence family, produces a new goodness-of-fit test statistic that can be a very attractive alternative to the Pearson's chi-square. Some numerical results suggest that, application of this test statistic, as an alternative to Pearson's chi-square, could be preferable to the I ^2/3 statistic of Cressie and Read (1984) under the use of chi-square critical values.     

Fast computation of the exact null distribution of Spearman's ρ and Page's L statistic for samples with and without ties

We present a new algorithm for computing the exact null distribution of the Spearman rank correlation statistic ρ, which also works in the case of ties. The algorithm is based on symmetries in the representation of the probability generating function as a permanent with monomial entries. We present new critical values for sample sizes 19⩽n⩽22. Finally, we show how to derive the exact null distribution of Page's L statistic from the null distribution of ρ.     

A goodness-of-fit test for the stratified proportional hazards model for survival data

Abstract

Goodness-of-fit testing is addressed in the stratified proportional hazards model for survival data. A test statistic based on within-strata cumulative sums of martingale residuals over covariates is proposed and its asymptotic distribution is derived under the null hypothesis of model adequacy. A Monte Carlo procedure is proposed to approximate the critical value of the test. Simulation studies are conducted to examine finite-sample performance of the proposed statistic.     

Percentage points of LRC for testing equality of covariance matrices under intraclass correlation structure

The exact null distribution of the likelihood ratio test statistic for testing equality of covariance matrices of q compound symmetric Gaussian models (bivariate or trivariate) has been obtained and percentage points for q ? 5 have been computed. The inverse Mellin transform and calculus of residues have been used to derive these results.     

Testing a covariance matrix: exact null distribution of its likelihood criterion

For X～ N _p (μ, Σ) testing H _o:Σ = Σ ₀, with Σ ₀ known, relies at present on an approximation of the null-distribution of the likelihood ratio statistic.

We present here the exact null distribution and also its computation, hence providing a precise tool that can be used in small sample cases.     

An Efficient Method to Generate Data and Compute Exact P-Values in Goodness-of-Fit Testing

In this article, we use a characterization of the set of sample counts that do not match with the null hypothesis of the test of goodness of fit. Two direct applications arise: first, to instantaneously generate data sets whose corresponding asymptotic P-values belong to a certain pre-defined range; and second, to compute exact P-values for this test in an efficient way. We present both issues before illustrating them by analyzing a couple of data sets. Method's efficiency is also assessed by means of simulations. We focus on Pearson's X ² statistic but the case of likelihood-ratio statistic is also discussed.     

A Laguerre polynomial approximation for a goodness-of-fit test for exponential distribution based on progressively censored data

This paper proposes an approximation to the distribution of a goodness-of-fit statistic proposed recently by Balakrishnan et al. [Balakrishnan, N., Ng, H.K.T. and Kannan, N., 2002, A test of exponentiality based on spacings for progressively Type-II censored data. In: C. Huber-Carol et al. (Eds.), Goodness-of-Fit Tests and Model Validity (Boston: Birkhäuser), pp. 89–111.] for testing exponentiality based on progressively Type-II right censored data. The moments of this statistic can be easily calculated, but its distribution is not known in an explicit form. We first obtain the exact moments of the statistic using Basu's theorem and then the density approximants based on these exact moments of the statistic, expressed in terms of Laguerre polynomials, are proposed. A comparative study of the proposed approximation to the exact critical values, computed by Balakrishnan and Lin [Balakrishnan, N. and Lin, C.T., 2003, On the distribution of a test for exponentiality based on progressively Type-II right censored spacings. Journal of Statistical Computation and Simulation, 73 (4), 277–283.], is carried out. This reveals that the proposed approximation is very accurate.     

INFLUENCE FUNCTION OF THE LIKELIHOOD RATIO TEST STATISTIC FOR MULTIVARIATE NORMAL SAMPLE

ABSTRACT

We derive the influence function of the likelihood ratio test statistic for multivariate normal sample. The derived influence function does not depend on the influence functions of the parameters under the null hypothesis. So we can obtain directly the empirical influence function with only the maximum likelihood estimators under the null hypothesis. Since the derived formula is a general form, it can be applied to influence analysis on many statistical testing problems.     

Continuity corrected approximations for and ‘exact’ inference with Pearson's X2

The classical adjustments for the inadequacy of the asymptotic distribution of Pearson's X² statistic, when some cells are sparse or the cell expectations are small, use continuity corrections and exact moments; the recent approach is to use computer based ‘exact inference’. In this paper we observe that the original exact test due to Freeman and Halton (Biometrika 38 (1951), 141–149) and its computer implementation are theoretically unsound. Furthermore, the corrected algorithmic version for the exact p-value in StatXact is practically useful in very few cases, and the results of its present version which includes Monte Carlo estimates can be highly variable. We then derive asymptotic expansions for the moments of the null distribution of Pearson's X², introduce a new method of correcting for discreteness and finite range of Pearson's X² as an alternative to the classical continuity correction, and use them to construct new and improved approximations for the null distribution. We also offer diagnostic criteria applicable to the tables for selecting an appropriate approximation. The exact methods and the competing approximations are studied and compared using thirteen test cases from the literature. It is concluded that the accuracy of the appropriate approximation is comparable with the truly exact method whenever it is available. The use of approximations is therefore preferable if the truly exact computer intensive solutions are unavailable or infeasible.     

Outlier detection in contingency tables using decomposable graphical models

For high-dimensional data, it is a tedious task to determine anomalies such as outliers. We present a novel outlier detection method for high-dimensional contingency tables. We use the class of decomposable graphical models to model the relationship among the variables of interest, which can be depicted by an undirected graph called the interaction graph. Given an interaction graph, we derive a closed-form expression of the likelihood ratio test (LRT) statistic and an exact distribution for efficient simulation of the test statistic. An observation is declared an outlier if it deviates significantly from the approximated distribution of the test statistic under the null hypothesis. We demonstrate the use of the LRT outlier detection framework on genetic data modeled by Chow–Liu trees.     

Asymptotic properties of the bootstrap unit root test statistic under possibly infinite variance

ABSTRACT

In this article, the unit root test for the AR(1) model is discussed, under the condition that the innovations of the model are in the domain of attraction of the normal law with possibly infinite variances. By using residual bootstrap with sample size m < n (n being the size of the original sample), we bootstrap the least-squares estimator of the autoregressive parameter. Under some mild assumptions, we prove that the null distribution of the unit root test statistic based on the least-square estimator of the autoregressive parameter can be approximated by using residual bootstrap.     

A Proposed Hybrid Effect Size Plus p-Value Criterion: Empirical Evidence Supporting its Use

ABSTRACT

When the editors of Basic and Applied Social Psychology effectively banned the use of null hypothesis significance testing (NHST) from articles published in their journal, it set off a fire-storm of discussions both supporting the decision and defending the utility of NHST in scientific research. At the heart of NHST is the p-value which is the probability of obtaining an effect equal to or more extreme than the one observed in the sample data, given the null hypothesis and other model assumptions. Although this is conceptually different from the probability of the null hypothesis being true, given the sample, p-values nonetheless can provide evidential information, toward making an inference about a parameter. Applying a 10,000-case simulation described in this article, the authors found that p-values’ inferential signals to either reject or not reject a null hypothesis about the mean (α?=?0.05) were consistent for almost 70% of the cases with the parameter’s true location for the sampled-from population. Success increases if a hybrid decision criterion, minimum effect size plus p-value (MESP), is used. Here, rejecting the null also requires the difference of the observed statistic from the exact null to be meaningfully large or practically significant, in the researcher’s judgment and experience. The simulation compares performances of several methods: from p-value and/or effect size-based, to confidence-interval based, under various conditions of true location of the mean, test power, and comparative sizes of the meaningful distance and population variability. For any inference procedure that outputs a binary indicator, like flagging whether a p-value is significant, the output of one single experiment is not sufficient evidence for a definitive conclusion. Yet, if a tool like MESP generates a relatively reliable signal and is used knowledgeably as part of a research process, it can provide useful information.     

An exact test for trend among binomial proportions based on a modified Baumgartner-Weiß-Schindler statistic

The Cochran-Armitage test is the most frequently used test for trend among binomial proportions. This test can be performed based on the asymptotic normality of its test statistic or based on an exact null distribution. As an alternative, a recently introduced modification of the Baumgartner-Weiß-Schindler statistic, a novel nonparametric statistic, can be used. Simulation results indicate that the exact test based on this modification is preferable to the Cochran-Armitage test. This exact test is less conservative and more powerful than the exact Cochran-Armitage test. The power comparison to the asymptotic Cochran-Armitage test does not show a clear winner, but the difference in power is usually small. The exact test based on the modification is recommended here because, in contrast to the asymptotic Cochran-Armitage test, it guarantees a type I error rate less than or equal to the significance level. Moreover, an exact test is often more appropriate than an asymptotic test because randomization rather than random sampling is the norm, for example in biomedical research. The methods are illustrated with an example data set.     

More powerful sign test using median ranked set sample: Finite sample power comparison

Sign test using median ranked set samples (MRSS) is introduced and investigated. We show that, this test is more powerful than the sign tests based on simple random sample (SRS) and ranked set sample (RSS) for finite sample size. It is found that, when the set size of MRSS is odd, the null distribution of the MRSS sign test is the same as the sign test obtained by using SRS. The exact null distributions and the power functions, in case of finite sample sizes, of these tests are derived. Also, the asymptotic distribution of the MRSS sign tests are derived. Numerical comparison of the MRSS sign test power with the power of the SRS sign test and the RSS sign test is given. Illustration of the procedure, using real data set of bilirubin level in Jaundice babies who stay in neonatal intensive care is introduced.