A note on asymptotic distribution of permutation statistics from mixing processes

The permutation statistics from a-mixing processes with random, symmetric weights are studied The asymptotic distribution of the permutation statistics is derived.     

PERMUTATION TESTS AND OTHER TEST STATISTICS FOR ILL-BEHAVED DATA: EXPERIENCE OF THE NINDS t-PA STROKE TRIAL

Permutation tests are often used to analyze data since they may not require one to make assumptions regarding the form of the distribution to have a random and independent sample selection. We initially considered a permutation test to assess the treatment effect on computed tomography lesion volume in the National Institute of Neurological Disorders and Stroke (NINDS) t-PA Stroke Trial, which has highly skewed data. However, we encountered difficulties in summarizing the permutation test results on the lesion volume. In this paper, we discuss some aspects of permutation tests and illustrate our findings. This experience with the NINDS t-PA Stroke Trial data emphasizes that permutation tests are useful for clinical trials and can be used to validate assumptions of an observed test statistic. The permutation test places fewer restrictions on the underlying distribution but is not always distribution-free or an exact test, especially for ill-behaved data. Quasi-likelihood estimation using the generalized estimating equation (GEE) approach on transformed data seems to be a good choice for analyzing CT lesion data, based on both its corresponding permutation test and its clinical interpretation.     

Uniqueness of permuted treatment estimator distributions in the proportional hazards model

We discuss findings regarding the permutation distributions of treatment effect estimators in the proportional hazards model. For fixed sample size n, we will prove that all uncensored and untied event times yield the same permutation distribution of treatment effect estimators in the proportional hazards model. In other words this distribution is irrelevant with respect to the actual event times. We will show several uniqueness properties under different conditions. These properties are useful for small sample permutation tests and also helpful to large sample cases.     

A sequential approximation to a permutation test

A sequential method for approximating a general permutation test (SAPT) is proposed and evaluated. Permutations are randomly generated from some set G, and a sequential probability ratio test (SPRT) is used to determine whether an observed test statistic falls sufficiently far in the tail of the permutation distribution to warrant rejecting some hypothesis. An estimate and bounds on the power function of the SPRT are used to find bounds on the effective significance level of the SAPT. Guidelines are developed for choosing parameters in order to obtain a desired significance level and minimize the number of permutations needed to reach a decision. A theoretical estimate of the average number of permutations under the null hypothesis is given along with simulation results demonstrating the power and average number of permutations for various alternatives. The sequential approximation retains the generality of the permutation test,- while avoiding the computational complexities that arise in attempting to computer the full permutation distribution exactly     

Large sample results for permutation tests of association

This paper deals with the asymptotics of permutation tests based on a certain rather general class of measures of association for R by C contingency tables, given marginal totals. This class includes the classical chi-square test, the T _b and γ indices of Goodman and Kruskall (1954) and the popular Rand (1971) index. The asymptotic distribution of this class of permutation tests for association is a weighted sum of non-central (gen-erally speaking) chi-squares. The formulae for the asymptotic moments of such tests are also given. If non-centrality holds under the null hypothe-sis of independence, the distribution in question converges to the normal distribution. The efficacies for such measures of association are obtained. Several applications are analysed in detail, including the above mentioned indices. Approximations to the permutation distribution are also discussed.     

A toolbox of permutation tests for structural change

The sup $LM$ test for structural change is embedded into a permutation test framework for a simple location model. The resulting conditional permutation distribution is compared to the usual (unconditional) asymptotic distribution, showing that the power of the test can be clearly improved in small samples. Furthermore, the permutation test is embedded into a general framework that encompasses tools for binary and multivariate dependent variables as well as model-based permutation testing for structural change. It is also demonstrated that the methods can not only be employed for analyzing structural changes in time series data but also for recursive partitioning of cross-section data. The procedures suggested are illustrated using both artificial data and empirical applications (number of youth homicides, employment discrimination data, carbon flux in tropical forests, stock returns, and demand for economics journals).     

APPROXIMATIONS TO SOME TEST STATISTICS FOR PERMUTATION TESTS IN A COMPLETELY RANDOMIZED DESIGN1

Under a randomization model for a completely randomized design permutation tests are considered based on the usual F statistic and on a multi-response permutation procedure statistic. For the first statistic the first two moments are obtained so a comparision with the distribution under the normal theory model can be made. The second statistic is shown to converge in distribution to an infinite weighted sum of chi-squared variates, the weights being the limits of the eigenvalues of a matrix depending on the distance measure used and the order statistics of the observations.     

Fourth exact moment results for mrbp tests with two or three treatments

MRBP tests were proposed by Mielke and Iyer (1982) to analyze multivariate data for the randomized block design, based on permutation procedures. They obtained the first three exact moments of the MRBP test statistic to approximate its permutation distribution. Tracy and Khan (1991) derived its fourth exact moment, to obtain a better approximating distribution, when there are four or more treatments. In this paper we obtain the fourth exact moment when the number of treatments is less than four.     

Exact permutation inference for two sample repeated measures data

Experiments in which very few units are measured many times sometimes present particular difficulties. Interest often centers on simple location shifts between two treatment groups, but appropriate modeling of the error distribution can be challenging. For example, normality may be difficult to verify, or a single transformation stabilizing variance or improving normality for all units and all measurements may not exist. We propose an analysis of two sample repeated measures data based on the permutation distribution of units. This provides a distribution free alternative to standard analyses. The analysis includes testing, estimation and confidence intervals. By assuming a certain structure in the location shift model, the dimension of the problem is reduced by analyzing linear combinations of the marginal statistics. Recently proposed algorithms for computation of two sample permutation distributions, require only a few seconds for experiments having as many as 100 units and any number of repeated measures. The test has high asymptotic efficiency and good power with respect to tests based on the normal distribution. Since the computational burden is minimal, approximation of the permutation distribution is unnecessary.     

A review and some new results on permutation testing for multivariate problems

In recent years permutation testing methods have increased both in number of applications and in solving complex multivariate problems. When available permutation tests are essentially of an exact nonparametric nature in a conditional context, where conditioning is on the pooled observed data set which is often a set of sufficient statistics in the null hypothesis. Whereas, the reference null distribution of most parametric tests is only known asymptotically. Thus, for most sample sizes of practical interest, the possible lack of efficiency of permutation solutions may be compensated by the lack of approximation of parametric counterparts. There are many complex multivariate problems, quite common in empirical sciences, which are difficult to solve outside the conditional framework and in particular outside the method of nonparametric combination (NPC) of dependent permutation tests. In this paper we review such a method and its main properties along with some new results in experimental and observational situations (robust testing, multi-sided alternatives and testing for survival functions).     

Bivariate symmetry tests for complete and competing risks data: a saddlepoint approach

A class of bivariate symmetry tests for complete data and competing risks data is considered. Saddlepoint approximation for the exact p-values of the underlying permutation distribution of these tests is derived. Several simulation studies are conducted to evaluate the performance of the saddlepoint approximation and the asymptotic approximation. The saddlepoint approximation was found to be highly accurate and superior to the asymptotic approximations in replicating the exact permutation significance.     

Distribution-free tests for cluster samples of ordinal responses

A simulation comparison is done of Mann–Whitney U test extensions recently proposed for simple cluster samples or for repeated ordinal responses. These are based on two approaches: the permutation approach of Fay and Gennings (four tests, two exact), and Edwardes’ approach (two asymptotic tests, one new). Edwardes’ approach permits confidence interval estimation, unlike the permutation approach. An appropriate parameter for estimation is P(X<Y)−P(X>Y), where X is the rank of a response from group 1 and Y is from group 2. The permutation tests are shown to be unsuitable for some survey data, since they are sensitive to a difference in cluster intra-correlations when there is no distribution difference between groups at the individual level. The exact permutation tests are of little use for less than seven clusters, precisely where they are most needed. Otherwise, the permutation tests perform well.     

Exact distribution-free tests for equality of several linear models

Exact tests for the equality of several linear models are developed using permutation techniques. Two cases of the linear model, characterized by either stochastic or nonstochastic predictors, are considered: the linear regression model (LRM) and the general linear model (GLM). A general class of test statistics using the volume of simplexes as the basic unit of analysis is proposed for this problem. The resulting class of statistics is shown to be a natural generalization of the multi-response permutation procedure (MRPP) test statistics which have been shown to comprise many of the statistics used in both parametric and nonparametric analysis of the standard g—sample problem. In the LRM case, exact moments of all orders are derived for the permutation distribution of any test statistic in the general class. Moment-based approximation of significance levels is shown to be computationally feasible in the simple LRM.     

Fourth moment results for mrbp and related power performance

A permutation test for analysing randomized block data was proposed by Mielke and Iyer (1982). They obtained the first three exact moments of this test statistic and approximated its permutation distribution by the Pearson type III distribution. Tracy and Khan (1991) derived the fourth exact moment of this test statistic to obtain a better approximating distribution. Here we obtain the simplified form of the fourth moment result for some special cases of this test statistic. Empirical powers for four treatments are compared, using this additional information, with those based on the three moment results, after simulating data from some underlying populations.     

Bootstrapping Noncausal Autoregressions: With Applications to Explosive Bubble Modeling

In this article, we develop new bootstrap-based inference for noncausal autoregressions with heavy-tailed innovations. This class of models is widely used for modeling bubbles and explosive dynamics in economic and financial time series. In the noncausal, heavy-tail framework, a major drawback of asymptotic inference is that it is not feasible in practice as the relevant limiting distributions depend crucially on the (unknown) decay rate of the tails of the distribution of the innovations. In addition, even in the unrealistic case where the tail behavior is known, asymptotic inference may suffer from small-sample issues. To overcome these difficulties, we propose bootstrap inference procedures using parameter estimates obtained with the null hypothesis imposed (the so-called restricted bootstrap). We discuss three different choices of bootstrap innovations: wild bootstrap, based on Rademacher errors; permutation bootstrap; a combination of the two (“permutation wild bootstrap”). Crucially, implementation of these bootstraps do not require any a priori knowledge about the distribution of the innovations, such as the tail index or the convergence rates of the estimators. We establish sufficient conditions ensuring that, under the null hypothesis, the bootstrap statistics estimate consistently particular conditionaldistributions of the original statistics. In particular, we show that validity of the permutation bootstrap holds without any restrictions on the distribution of the innovations, while the permutation wild and the standard wild bootstraps require further assumptions such as symmetry of the innovation distribution. Extensive Monte Carlo simulations show that the finite sample performance of the proposed bootstrap tests is exceptionally good, both in terms of size and of empirical rejection probabilities under the alternative hypothesis. We conclude by applying the proposed bootstrap inference to Bitcoin/USD exchange rates and to crude oil price data. We find that indeed noncausal models with heavy-tailed innovations are able to fit the data, also in periods of bubble dynamics. Supplementary materials for this article are available online.     

A Permutation Test for Planar Regression

Oja (1987) presents some distribution-free tests applicable in the presence of covariates when treatment values are randomly assigned. The formulas and calculations are cumbersome, however, and implementation of the tests relies on using a x² approximation to the exact null distribution. In this paper a re-formulation of his test statistic is given which has the advantages of ease of calculation, explicit formulas for permutation moments, and allowing a Beta distribution to be fitted to the exact null distribution.     

A NONPARAMETRIC TEST FOR THE EQUALITY OF DEPENDENT CORRELATION COEFFICIENTS UNDER NORMALITY

ABSTRACT

A nonparametric testing method for the equality of two correlation coefficients in trivariate normal distribution, namely, one of the variables are common, is discussed. Using a permutation test, we obtain asymptotically exact solutions. The performance of this test is compared with the likelihood ratio test and a method of using the limiting distribution of correlation coefficients.     

The split sample permutation t-tests

Without the exchangeability assumption, permutation tests for comparing two population means do not provide exact control of the probability of making a Type I error. Another drawback of permutation tests is that it cannot be used to test hypothesis about one population. In this paper, we propose a new type of permutation tests for testing the difference between two population means: the split sample permutation t-tests. We show that the split sample permutation t-tests do not require the exchangeability assumption, are asymptotically exact and can be easily extended to testing hypothesis about one population. Extensive simulations were carried out to evaluate the performance of two specific split sample permutation t-tests: the split in the middle permutation t-test and the split in the end permutation t-test. The simulation results show that the split in the middle permutation t-test has comparable performance to the permutation test if the population distributions are symmetric and satisfy the exchangeability assumption. Otherwise, the split in the end permutation t-test has significantly more accurate control of level of significance than the split in the middle permutation t-test and other existing permutation tests.     

Analyzing the results of a cloud-seeding experiment in tasmania

We considered the analysis or a randomized cloud seeding experiment of lusmania, Where an distributed analysis had been specified before the experiment was carried one. We compare “classical” regression analyses, with and without transformations, with permutation tests based upon double-ratio and variance-ratio statistics. Regression residuals are used to compare the merits of various other alternative test-statistics including some based upon gamma distribution assumptions. We conclude that more attention needs to be paid to the relative weights which various rest statistics implicitly attach to low, medium and high rainfalls, and that, irrespective ot distributional assumptions and theory, significant testing should be done using permutation tests.     

Urn Sampling and the Proportional Hazard Model

This paper investigates the urn sampling analogue for the score statistic relating survival to covariates assuming a proportional hazard model. The exact permutation distribution can be calculated as well as the exact low order moments for arbitrary censoring patterns. The asymptotic distribution of the score statistic is an easy consequence. The method is naturally extended to deal with the multivariate case, time varying covariates and interval censoring. Finally the relationship between the censoring process, the survival times and covariates are studied considering different reference sets for the distribution of the score statistic. Some assumptions about the censoring process are investigated and as a consequence the effect of censoring is clarified.