Extending permutation conditional inference to unconditional ones

In this presentation we discuss the extension of permutation conditional inferences to unconditional or population ones. Within the parametric approach this extension is possible when the data set is randomly selected by well-designed sampling procedures on well-defined population distributions, provided that their nuisance parameters have boundely complete statistics in the null hypothesis or are provided with invariant statistics. When these conditions fail, especially if selection-bias procedures are used for data collection processes, in general most of the parametric inferential extensions are wrong or misleading. We will see that, since they are provided with similarity and conditional unbiasedness properties and if correctly applicable, permutation tests may extend, at least in a weak sense, conditional to unconditional inferences.     

The split sample permutation t-tests

Without the exchangeability assumption, permutation tests for comparing two population means do not provide exact control of the probability of making a Type I error. Another drawback of permutation tests is that it cannot be used to test hypothesis about one population. In this paper, we propose a new type of permutation tests for testing the difference between two population means: the split sample permutation t-tests. We show that the split sample permutation t-tests do not require the exchangeability assumption, are asymptotically exact and can be easily extended to testing hypothesis about one population. Extensive simulations were carried out to evaluate the performance of two specific split sample permutation t-tests: the split in the middle permutation t-test and the split in the end permutation t-test. The simulation results show that the split in the middle permutation t-test has comparable performance to the permutation test if the population distributions are symmetric and satisfy the exchangeability assumption. Otherwise, the split in the end permutation t-test has significantly more accurate control of level of significance than the split in the middle permutation t-test and other existing permutation tests.     

Exact permutation inference for two sample repeated measures data

Experiments in which very few units are measured many times sometimes present particular difficulties. Interest often centers on simple location shifts between two treatment groups, but appropriate modeling of the error distribution can be challenging. For example, normality may be difficult to verify, or a single transformation stabilizing variance or improving normality for all units and all measurements may not exist. We propose an analysis of two sample repeated measures data based on the permutation distribution of units. This provides a distribution free alternative to standard analyses. The analysis includes testing, estimation and confidence intervals. By assuming a certain structure in the location shift model, the dimension of the problem is reduced by analyzing linear combinations of the marginal statistics. Recently proposed algorithms for computation of two sample permutation distributions, require only a few seconds for experiments having as many as 100 units and any number of repeated measures. The test has high asymptotic efficiency and good power with respect to tests based on the normal distribution. Since the computational burden is minimal, approximation of the permutation distribution is unnecessary.     

A Class of Permutation Tests for the Equality of Two Marginal Survival Functions Using Paired Censored Data

We studied several test statistics for testing the equality of marginal survival functions of paired censored data. The null distribution of the test statistics was approximated by permutation. These tests do not require explicit modeling or estimation of the within-pair correlation, accommodate both paired data and singletons, and the computation is straightforward with most statistical software. Numerical studies showed that these tests have competitive size and power performance. One test statistic has higher power than previously published test statistics when the two survival functions under comparison cross. We illustrate use of these tests in a propensity score matched dataset.     

Combination-Based Permutation Tests: Equipower Property and Power Behavior in Presence of Correlation

Multivariate combination-based permutation tests have been widely used in many complex problems. In this paper we focus on the equipower property, derived directly from the finite-sample consistency property, and we analyze the impact of the dependency structure on the combined tests. At first, we consider the finite-sample consistency property which assumes that sample sizes are fixed (and possibly small) and considers on each subject a large number of informative variables. Moreover, since permutation test statistics do not require to be standardized, we need not assume that data are homoscedastic in the alternative. The equipower property is then derived from these two notions: consider the unconditional permutation power of a test statistic T for fixed sample sizes, with V ? 2 independent and identically distributed variables and fixed effect δ, calculated in two ways: (i) by considering two V-dimensional samples sized m₁ and m₂, respectively; (ii) by considering two unidimensional samples sized n₁ = Vm₁ and n₂ = Vm₂, respectively. Since the unconditional power essentially depends on the non centrality induced by T, and two ways are provided with exactly the same likelihood and the same non centrality, we show that they are provided with the same power function, at least approximately. As regards both investigating the equipower property and the power behavior in presence of correlation we performed an extensive simulation study.     

PERMUTATION TESTS AND OTHER TEST STATISTICS FOR ILL-BEHAVED DATA: EXPERIENCE OF THE NINDS t-PA STROKE TRIAL

Permutation tests are often used to analyze data since they may not require one to make assumptions regarding the form of the distribution to have a random and independent sample selection. We initially considered a permutation test to assess the treatment effect on computed tomography lesion volume in the National Institute of Neurological Disorders and Stroke (NINDS) t-PA Stroke Trial, which has highly skewed data. However, we encountered difficulties in summarizing the permutation test results on the lesion volume. In this paper, we discuss some aspects of permutation tests and illustrate our findings. This experience with the NINDS t-PA Stroke Trial data emphasizes that permutation tests are useful for clinical trials and can be used to validate assumptions of an observed test statistic. The permutation test places fewer restrictions on the underlying distribution but is not always distribution-free or an exact test, especially for ill-behaved data. Quasi-likelihood estimation using the generalized estimating equation (GEE) approach on transformed data seems to be a good choice for analyzing CT lesion data, based on both its corresponding permutation test and its clinical interpretation.     

The Use of Permutation Tests for Variance Components in Linear Mixed Models

Standard asymptotic chi-square distribution of the likelihood ratio and score statistics under the null hypothesis does not hold when the parameter value is on the boundary of the parameter space. In mixed models it is of interest to test for a zero random effect variance component. Some available tests for the variance component are reviewed and a new test within the permutation framework is presented. The power and significance level of the different tests are investigated by means of a Monte Carlo simulation study. The proposed test has a significance level closer to the nominal one and it is more powerful.     

Permuiation tesis for k-sample binomial data with comparisons of exact and approximate P-levels

Exact ksample permutation tests for binary data for three commonly encountered hypotheses tests are presented,, The tests are derived both under the population and randomization models . The generating function for the number of cases in the null distribution is obtained, The asymptotic distributions of the test statistics are derived . Actual significance levels are computed for the asymptotic test versions , Random sampling of the null distribution is suggested as a superior alternative to the asymptotics and an efficient computer technique for implementing the random sampling is described., finally, some numerical examples are presented and sample size guidelines given for computer implementation of the exact tests.     

Statistical tests for random effects in staggered nested designs

In this paper, we present certain statistical tests under a staggered nested design set-up, for the hypotheses that certain variance components are zero. To do so, the particular variance-covariance, structure induced by the staggering is exploited and certain results of multivariate analysis are used. In most problems, the test statistics can be easily computed. An example is provided for illustration and some power computations for comparison of test statistics are shown.     

Robust permutation tests for two samples

We consider robust permutation tests for a location shift in the two sample case based on estimating equations, comparing the test statistics based on a score function and an M-estimate. First we obtain a form for both tests so that the exact tests may be carried out using the same algorithms as used for permutation tests based on the mean. Then we obtain the Bahadur slopes of the tests in these two statistics, giving numerical results for two cases equivalent to a test based on Huber scores and a particular case of this related to a median test. We show that they have different Bahadur slopes with neither exceeding the other over the whole range. Finally, we give some numerical results illustrating the robustness properties of the tests and confirming the theoretical results on Bahadur slopes.     

Multi-response permutation procedures for a priori classifications

An exact permutation test for analyzing and/or dredging multi-response data at the ordinal or higher levels is presented. The associated test statistic is based on the average distance (or any specified norm) between points within a priori disjoint subgroups of a finite population of points in an r-dimensional space (corresponding to r measured responses from each object in a finite population of objects). Alternative approximate tests based on the beta and normal distributions are provided. Two detailed examples utilizing actual social science data are considered, including comparisons of the approximate tests. An additional example describes the behavior of these tests under a variety of conditions, including extreme data configurations     

Monte carlo results for conditional survival tests under randomly censored data

Within a Monte Carlo study finite sample results are obtained for different generalized rank tests based on randomly censored life time data. It is pointed out that conditional tests should be applied in practice whenever drastic differences between the censoring distributions for the underlying groups do not appear. The tests are slight modifications of known permutation tests for censored data.     

Detecting parameter shift in garch models

This paper applies recent theories of testing for parameter constancy to the conditional variance in a GARCH model. The supremum Lagrange multiplier test for conditional Gaussian GARCH models and its robustified variants are discussed. The asymptotic null distribution of the test statistics are derived from the weak convergence of the scores, and the critical values from the hitting probability of squared Bessel process.

Monte Carlo studies on the finite sample size and power performance of the supremum LM tests are conducted. Applications of these tests to S&P 500 indicate that the hypothesis of stable conditional variance parameters can be rejected.     

Detecting parameter shift in garch models

This paper applies recent theories of testing for parameter constancy to the conditional variance in a GARCH model. The supremum Lagrange multiplier test for conditional Gaussian GARCH models and its robustified variants are discussed. The asymptotic null distribution of the test statistics are derived from the weak convergence of the scores, and the critical values from the hitting probability of squared Bessel process.

Monte Carlo studies on the finite sample size and power performance of the supremum LM tests are conducted. Applications of these tests to S&P 500 indicate that the hypothesis of stable conditional variance parameters can be rejected.     

Exact Poly-K Test

For animal carcinogenicity study with multiple dose groups, positive trend test and pairwise comparisons of treated groups with control are generally performed using the Cochran-Armitage, Peto test, or Poly-K test. These tests are asymptotically normal. The exact version of Cochran-Armitage and Peto tests are available based on the permutation test assuming fixed column and row totals. For Poly-K test column totals depend on the mortality pattern of the animals and can not be kept fixed over the permutations of the animals. In this work a modification of the permutation test is suggested that can be applied on exact Poly-K test.     

Selection between models through multi-step-ahead forecasting

We develop and show applications of two new test statistics for deciding if one ARIMA model provides significantly better h-step-ahead forecasts than another, as measured by the difference of approximations to their asymptotic mean square forecast errors. The two statistics differ in the variance estimates used for normalization. Both variance estimates are consistent even when the models considered are incorrect. Our main variance estimate is further distinguished by accounting for parameter estimation, while the simpler variance estimate treats parameters as fixed. Their broad consistency properties offer improvements to what are known as tests of Diebold and Mariano (1995) type, which are tests that treat parameters as fixed and use variance estimates that are generally not consistent in our context. We show how these statistics can be calculated for any pair of ARIMA models with the same differencing operator.     

Detecting Variance Change-Points for Blocked Time Series and Dependent Panel Data

This article proposes a class of weighted differences of averages (WDA) statistics to test and estimate possible change-points in variance for time series with weakly dependent blocks and dependent panel data without specific distributional assumptions. We derive the asymptotic distributions of the test statistics for testing the existence of a single variance change-point under the null and local alternatives. We also study the consistency of the change-point estimator. Within the proposed class of the WDA test statistics, a standardized WDA test is shown to have the best consistency rate and is recommended for practical use. An iterative binary searching procedure is suggested for estimating the locations of possible multiple change-points in variance, whose consistency is also established. Simulation studies are conducted to compare detection power and number of wrong rejections of the proposed procedure to that of a cumulative sum (CUSUM) based test and a likelihood ratio-based test. Finally, we apply the proposed method to a stock index dataset and an unemployment rate dataset. Supplementary materials for this article are available online.     

Block permutation principles for the change analysis of dependent data

We study an AMOC model with an abrupt change in the mean and dependent errors that form a linear process. Different kinds of statistics are considered, such as maximum-type statistics (particularly different CUSUM procedures) or sum-type statistics. Approximations of the critical values for change-point tests are obtained through permutation methods. The theoretical results show that the original test statistics and their corresponding block permutation counterparts follow the same distributional asymptotics. The main step in the proof is to obtain limit theorems for the corresponding rank statistics and then use laws of large numbers to obtain the permutation asymptotics conditionally on the given data.     

Asymptotic normality of MRPP statistics from invariance principles of u-statistics

Multi-response permutation procedures (MRPP) were recently introduced to test differences between a priori classified groups of objects ( Mielke, Berry Johnson, 1976; Mielke, 1979 ). The null distributions of the MRPP statistics were initially conjectured to be asymptotically normal for some specified conditions within the setting of a sequence of finite populations due to Madow ( 1948 ).

Asymptotic normality of a class of MRPP statistics (under the null hypothesis) is shown in two cases: (i) the setting which considers the populations to be the samples resulting from sequential independent identically distributed (i.i.d.) sampling (sampling from infinite populations) and (ii) the setting of a sequence of increasingly large finite populations (sampling from finite populations). The results are direct applications of the weak convergence of a U-statistic process in the i.i.d. case to a Brownian motion (Bhattacharyya and Sen, 1977) and of the weak convergence of a U-statistic process in the finite populations case to a Brownian bridge (Sen, 1972). The conditions are milder for the i.i.d. case than for the finite populations case. However, neither case provides a restriction of a practical consequence in applications of MRPP. In either case, convergence is shown to depend on the asymptotic ratios of the group sizes to the population size.     

Properties of a randomization test for multifactor comparisons of groups

Multivariate hypothesis testing in studies of vegetation is likely to be hindered by unrealistic assumptions when based on conventional statistical methods. This can be overcome by randomization tests. In this paper, the accuracy and power of a MANOVA randomization test are evaluated for one and two factors with interaction with simulated data from three distributions. The randomization test is based on the partitioning of sum of squares computed from Euclidean distances. In one-factor designs, sample size and variance inequality were evaluated. The results showed a high level of accuracy. The power curve was higher with normal distribution, lower with uniform, intermediate with lognormal and was sensitive to variance inequality. In two-factor designs, three methods of permutations and two statistics were compared. The results showed that permutation of the residuals with F _pseudo is accurate and can give good power for testing the interaction and restricted permutation for testing main factors.