Large sample results for permutation tests of association

This paper deals with the asymptotics of permutation tests based on a certain rather general class of measures of association for R by C contingency tables, given marginal totals. This class includes the classical chi-square test, the T _b and γ indices of Goodman and Kruskall (1954) and the popular Rand (1971) index. The asymptotic distribution of this class of permutation tests for association is a weighted sum of non-central (gen-erally speaking) chi-squares. The formulae for the asymptotic moments of such tests are also given. If non-centrality holds under the null hypothe-sis of independence, the distribution in question converges to the normal distribution. The efficacies for such measures of association are obtained. Several applications are analysed in detail, including the above mentioned indices. Approximations to the permutation distribution are also discussed.     

Weighted Logrank Permutation Tests for Randomly Right Censored Life Science Data

In biomedical research, weighted logrank tests are frequently applied to compare two samples of randomly right censored survival times. We address the question how to combine a number of weighted logrank statistics to achieve good power of the corresponding survival test for a whole linear space or cone of alternatives, which are given by hazard rates. This leads to a new class of semiparametric projection tests that are motivated by likelihood ratio tests for an asymptotic model. We show that these tests can be carried out as permutation tests and discuss their asymptotic properties. A simulation study together with the analysis of a classical data set illustrates the advantages.     

A survey of permutation tests for censored survival data

The present paper surveys tests with censored sur-vival data for 2-and k-samples and for association by a framework which classifies them into complete or restric-ted permutation tests and into tests based on U-or Savage-scores. The formulae of the resulting twelve tests are briefly described for quick reference. Some of the tests have been applied frequently in the past as the tests by Mantel, Breslow or Gehan; others have been developed rather recently, partly by the author. The concluding discussion presents the results of a simulation study, clarifying similarities and differences of the restricted and complete permutation approach, and deals with rela-tive efficiencies of the two scoring systems.     

Nonparametric tests for multivariate multi-sample locations based on data depth

Several nonparametric tests for multivariate multi-sample location problem are proposed in this paper. These tests are based on the notion of data depth, which is used to measure the centrality/outlyingness of a given point with respect to a given distribution or a data cloud. Proposed tests are completely nonparametric and implemented through the idea of permutation tests. Performance of the proposed tests is compared with existing parametric test and nonparametric test based on data depth. An extensive simulation study reveals that proposed tests are superior to the existing tests based on data depth with regard to power. Illustrations with real data are provided.     

Bayesian Network Structure Learning with Permutation Tests

In literature there are several studies on the performance of Bayesian network structure learning algorithms. The focus of these studies is almost always the heuristics the learning algorithms are based on, i.e., the maximization algorithms (in score-based algorithms) or the techniques for learning the dependencies of each variable (in constraint-based algorithms). In this article, we investigate how the use of permutation tests instead of parametric ones affects the performance of Bayesian network structure learning from discrete data. Shrinkage tests are also covered to provide a broad overview of the techniques developed in current literature.     

An extended class of permutation techniques for matched pairs

A class of matched-pairs permutation techniques based on distances between each pair of observed signed values is considered. Although many commonly-used inference techniques for matched pairs are members of this class, some of the more appealing inference techniques among this class have received very little attention. Two new simple rank tests of this class jointly possess both intuitive properties and location-alternative power characteristics which appear more appealing than the corresponding characteristics of either the sign test or the Wllcoxon signed-ranks test. In particular, power comparisons based on slmula-tions indicate that these new rank tests are jointly as good or even vastly superior to the sign test or the Wilcoxon signed-ranks test for location alternatives involving five symmetric distributions. The five distributions selected for these com-parisons include the Laplace, logistic, normal, uniform and a U-shaped distribution     

Exact Multivariate Permutation Tests for Fixed Effects in Mixed-Models

A test for the fixed effect in mixed-models is proposed. It is based on permutation strategy and is exact. The testing approach presented is very general and the class of models covered is very broad.

Multivariate responses with different type of variables (e.g., continuous, categorical, and ranks) are usually tested with separated models and the overall test are usually reached through Bonferroni-like combinations, i.e., without taking into account the joint distribution of the test statistics. On the contrary, in this approach the joint distribution is immediately obtained and the dependence among tests is taken into account in the overall test. The methods are implemented in the R package flip, freely available on CRAN.     

Exact distribution-free tests for equality of several linear models

Exact tests for the equality of several linear models are developed using permutation techniques. Two cases of the linear model, characterized by either stochastic or nonstochastic predictors, are considered: the linear regression model (LRM) and the general linear model (GLM). A general class of test statistics using the volume of simplexes as the basic unit of analysis is proposed for this problem. The resulting class of statistics is shown to be a natural generalization of the multi-response permutation procedure (MRPP) test statistics which have been shown to comprise many of the statistics used in both parametric and nonparametric analysis of the standard g—sample problem. In the LRM case, exact moments of all orders are derived for the permutation distribution of any test statistic in the general class. Moment-based approximation of significance levels is shown to be computationally feasible in the simple LRM.     

Dependency and truncated forms of combinations in multivariate combination-based permutation tests and ordered categorical variables

ABSTRACT

Quite an important problem usually occurs in several multi-dimensional hypotheses testing problems when variables are correlated. In this framework the non-parametric combination (NPC) of a finite number of dependent permutation tests is suitable to cover almost all real situations of practical interest since the dependence relations among partial tests are implicitly captured by the combining procedure itself without the need to specify them [Pesarin F, Salmaso L. Permutation tests for complex data: theory, applications and software. Chichester: Wiley; 2010a]. An open problem related to NPC-based tests is the impact of the dependency structure on combined tests, especially in the presence of categorical variables. This paper’s goal is firstly to investigate the impact of the dependency structure on the possible significance of combined tests in cases of ordered categorical responses using Monte Carlo simulations, then to propose some specific procedures aimed at improving the power of multivariate combination-based permutation tests. The results show that an increasing level of correlation/association among responses negatively affects the power of combination-based multivariate permutation tests. The application of special forms of combination functions based on the truncated product method [Zaykin DV, Zhivotovsky LA, Westfall PH, Weir BS. Truncated product method for combining p-values. Genet Epidemiol. 2002;22:170–185; Dudbridge F, Koeleman BPC. Rank truncated product of p-values, with application to genomewide association scans. Genet Epidemiol. 2003;25:360–366] or on Liptak combination allowed us, using Monte Carlo simulations, to demonstrate the possibility of mitigating the negative effect on power of combination-based multivariate permutation tests produced by an increasing level of correlation/association among responses.     

Conditional Studentized Survival Tests for Randomly Censored Models

It is shown that in the case of heterogenous censoring distributions Studentized survival tests can be carried out as conditional permutation tests given the order statistics and their censoring status. The result is based on a conditional central limit theorem for permutation statistics. It holds for linear test statistics as well as for sup-statistics. The procedure works under one of the following general circumstances for the two-sample problem: the unbalanced sample size case, highly censored data, certain non-convergent weight functions or under alternatives. For instance, the two-sample log rank test can be carried out asymptotically as a conditional test if the relative amount of uncensored observations vanishes asymptotically as long as the number of uncensored observations becomes infinite. Similar results hold whenever the sample sizes and are unbalanced in the sense that and hold.     

Saddlepoint p-values for group of linear rank tests

Many nonparametric tests in one sample problem, matched pairs, and competingrisks under censoring have the same underlying permutation distribution. This article proposes a saddlepoint approximation to the exact p-values of these tests instead of the asymptotic approximations. The performance of the saddlepoint approximation is assessed by using simulation studies that show the superiority of the saddlepoint methods over the asymptotic approximations in several settings. The use of the saddlepoint to approximate the p-values of class of two sample tests under complete randomized design is also discussed.     

Bivariate symmetry tests for complete and competing risks data: a saddlepoint approach

A class of bivariate symmetry tests for complete data and competing risks data is considered. Saddlepoint approximation for the exact p-values of the underlying permutation distribution of these tests is derived. Several simulation studies are conducted to evaluate the performance of the saddlepoint approximation and the asymptotic approximation. The saddlepoint approximation was found to be highly accurate and superior to the asymptotic approximations in replicating the exact permutation significance.     

A robust I-sample analysis of means type randomization test for variances for unbalanced designs

Two analysis of means type randomization tests for testing the equality of I variances for unbalanced designs are presented. Randomization techniques for testing statistical hypotheses can be used when parametric tests are inappropriate. Suppose that I independent samples have been collected. Randomization tests are based on shuffles or rearrangements of the (combined) sample. Putting each of the I samples ‘in a bowl’ forms the combined sample. Drawing samples ‘from the bowl’ forms a shuffle. Shuffles can be made with replacement (bootstrap shuffling) or without replacement (permutation shuffling). The tests that are presented offer two advantages. They are robust to non-normality and they allow the user to graphically present the results via a decision chart similar to a Shewhart control chart. A Monte Carlo study is used to verify that the permutation version of the tests exhibit excellent power when compared to other robust tests. The Monte Carlo study also identifies circumstances under which the popular Levene's test fails.     

Permutation techniques for analyzing multi-response data from randomized block experiments

A class of permutation techniques is presented for the randomized block design. This class is specifically devised for analyses involving multivariate data. A numerical example illustrates an application based on multivariate data. Many well known techniques are special cases of this class. Among these special cases are (i) the permutation version of the classical univariate technique for randomized blocks which 1s associated with analysis of variance, (ii) the Friedman randomized block test, (iii) one-sample matched-pair tests, (iv) the Pearson correlation measure, and (v) the Spearman rank correlation and foot-rule measures. Furthermore, variations and multivariate versions among this class suggest a variety of new techniques which have not received any previous attention.     

Analyzing the results of a cloud-seeding experiment in tasmania

We considered the analysis or a randomized cloud seeding experiment of lusmania, Where an distributed analysis had been specified before the experiment was carried one. We compare “classical” regression analyses, with and without transformations, with permutation tests based upon double-ratio and variance-ratio statistics. Regression residuals are used to compare the merits of various other alternative test-statistics including some based upon gamma distribution assumptions. We conclude that more attention needs to be paid to the relative weights which various rest statistics implicitly attach to low, medium and high rainfalls, and that, irrespective ot distributional assumptions and theory, significant testing should be done using permutation tests.     

The split sample permutation t-tests

Without the exchangeability assumption, permutation tests for comparing two population means do not provide exact control of the probability of making a Type I error. Another drawback of permutation tests is that it cannot be used to test hypothesis about one population. In this paper, we propose a new type of permutation tests for testing the difference between two population means: the split sample permutation t-tests. We show that the split sample permutation t-tests do not require the exchangeability assumption, are asymptotically exact and can be easily extended to testing hypothesis about one population. Extensive simulations were carried out to evaluate the performance of two specific split sample permutation t-tests: the split in the middle permutation t-test and the split in the end permutation t-test. The simulation results show that the split in the middle permutation t-test has comparable performance to the permutation test if the population distributions are symmetric and satisfy the exchangeability assumption. Otherwise, the split in the end permutation t-test has significantly more accurate control of level of significance than the split in the middle permutation t-test and other existing permutation tests.     

A REVIEW OF SYSTEMS COINTEGRATION TESTS

The literature on systems cointegration tests is reviewed and the various sets of assumptions for the asymptotic validity of the tests are compared within a general unifying framework. The comparison includes likelihood ratio tests, Lagrange multiplier and Wald type tests, lag augmentation tests, tests based on canonical correlations, the Stock-Watson tests and Bierens' nonparametric tests. Asymptotic results regarding the power of these tests and previous small sample simulation studies are discussed. Further issues and proposals in the context of systems cointegration tests are also considered briefly. New simulations are presented to compare the tests under uniform conditions. Special emphasis is given to the sensitivity of the test performance with respect to the trending properties of the DGP.     

A permutation test for randomness with power against smooth variation

A permutation test for the white noise hypothesis is described, offering power against a general class of smooth alternatives. Simulation results show that it performs well, as compared with similar tests available in the literature, in terms of power. An example demonstrates its use in a particular problem in which a test for randomness was sought without any specific alternative.     

PERMUTATION TEST FOR EQUALITY OF CORRELATION COEFFICIENTS IN TWO POPULATIONS

ABSTRACT

The purpose of this paper is to investigate the permutation tests for equality of correlation coefficients among two independent populations. We discuss how to apply permutation test to this problem and its asymptotic suitability. We also show some simulation studies and an example of the Iris data.     

Multivariate multi-sample tests for location based on data depth

A notion of data depth is used to measure centrality or outlyingness of a data point in a given data cloud. In the context of data depth, the point (or points) having maximum depth is called as deepest point (or points). In the present work, we propose three multi-sample tests for testing equality of location parameters of multivariate populations by using the deepest point (or points). These tests can be considered as extensions of two-sample tests based on the deepest point (or points). The proposed tests are implemented through the idea of Fisher's permutation test. Performance of earlier tests is studied by simulation. Illustration with two real datasets is also provided.