Robust permutation tests for one sample

We consider robust permutation tests based on an estimating equation comparing the test statistics based on the score function with those based on the M-estimate. First we obtain a form for the tests so that the exact tests may be carried out using the same algorithms as used for permutation tests based on the mean. Then we compare the efficiencies of the tests in two cases, equivalent to the sign test and a test based on Huber scores, showing that they are equivalent in the Pitman sense but that they have different Bahadur slopes with neither exceeding the other over the whole range.     

Robust permutation tests for two samples

We consider robust permutation tests for a location shift in the two sample case based on estimating equations, comparing the test statistics based on a score function and an M-estimate. First we obtain a form for both tests so that the exact tests may be carried out using the same algorithms as used for permutation tests based on the mean. Then we obtain the Bahadur slopes of the tests in these two statistics, giving numerical results for two cases equivalent to a test based on Huber scores and a particular case of this related to a median test. We show that they have different Bahadur slopes with neither exceeding the other over the whole range. Finally, we give some numerical results illustrating the robustness properties of the tests and confirming the theoretical results on Bahadur slopes.     

Foreword (and Preface)

We present new algorithms for computing the exact distributions and p-values of quadratic t-sample distribution-free statistics of Kruskal–Wallis type. These algorithms are presented in terms of generating functions. We show that our algorithm also works for cases with ties and that it is much faster than existing algorithms. Moreover, we show how to use the results for the Kruskal–Wallis type statistics to compute the exact null distribution of the Chacko–Shorack statistic.     

A permutation test for multivariate data with grouped components

In this paper, we consider a nonparametric test procedure for multivariate data with grouped components under the two sample problem setting. For the construction of the test statistic, we use linear rank statistics which were derived by applying the likelihood ratio principle for each component. For the null distribution of the test statistic, we apply the permutation principle for small or moderate sample sizes and derive the limiting distribution for the large sample case. Also we illustrate our test procedure with an example and compare with other procedures through simulation study. Finally, we discuss some additional interesting features as concluding remarks.     

Fast computation of the exact null distribution of Spearman's ρ and Page's L statistic for samples with and without ties

We present a new algorithm for computing the exact null distribution of the Spearman rank correlation statistic ρ, which also works in the case of ties. The algorithm is based on symmetries in the representation of the probability generating function as a permanent with monomial entries. We present new critical values for sample sizes 19⩽n⩽22. Finally, we show how to derive the exact null distribution of Page's L statistic from the null distribution of ρ.     

Block permutation principles for the change analysis of dependent data

We study an AMOC model with an abrupt change in the mean and dependent errors that form a linear process. Different kinds of statistics are considered, such as maximum-type statistics (particularly different CUSUM procedures) or sum-type statistics. Approximations of the critical values for change-point tests are obtained through permutation methods. The theoretical results show that the original test statistics and their corresponding block permutation counterparts follow the same distributional asymptotics. The main step in the proof is to obtain limit theorems for the corresponding rank statistics and then use laws of large numbers to obtain the permutation asymptotics conditionally on the given data.     

Permuiation tesis for k-sample binomial data with comparisons of exact and approximate P-levels

Exact ksample permutation tests for binary data for three commonly encountered hypotheses tests are presented,, The tests are derived both under the population and randomization models . The generating function for the number of cases in the null distribution is obtained, The asymptotic distributions of the test statistics are derived . Actual significance levels are computed for the asymptotic test versions , Random sampling of the null distribution is suggested as a superior alternative to the asymptotics and an efficient computer technique for implementing the random sampling is described., finally, some numerical examples are presented and sample size guidelines given for computer implementation of the exact tests.     

Saddlepoint p-values for a class of tests for comparing competing risks with censored data

One of the general problems in clinical trials and mortality rates is the comparison of competing risks. Most of the test statistics used for independent and dependent risks with censored data belong to the class of weighted linear rank tests in its multivariate version. In this paper, we introduce the saddlepoint approximations as accurate and fast approximations for the exact p-values of this class of tests instead of the asymptotic and permutation simulated calculations. Real data examples and extensive simulation studies showed the accuracy and stability performance of the saddlepoint approximations over different scenarios of lifetime distributions, sample sizes and censoring.     

Some nonparametric precedence-type tests based on progressively censored samples and evaluation of power

In this paper, we consider the problem of testing the equality of two distributions when both samples are progressively Type-II censored. We discuss the following two statistics: one based on the Wilcoxon-type rank-sum precedence test, and the second based on the Kaplan–Meier estimator of the cumulative distribution function. The exact null distributions of these test statistics are derived and are then used to generate critical values and the corresponding exact levels of significance for different combinations of sample sizes and progressive censoring schemes. We also discuss their non-null distributions under Lehmann alternatives. A power study of the proposed tests is carried out under Lehmann alternatives as well as under location-shift alternatives through Monte Carlo simulations. Through this power study, it is shown that the Wilcoxon-type rank-sum precedence test performs the best.     

Log‐rank permutation tests for trend: saddlepoint p‐values and survival rate confidence intervals

Suppose p + 1 experimental groups correspond to increasing dose levels of a treatment and all groups are subject to right censoring. In such instances, permutation tests for trend can be performed based on statistics derived from the weighted log‐rank class. This article uses saddlepoint methods to determine the mid‐P‐values for such permutation tests for any test statistic in the weighted log‐rank class. Permutation simulations are replaced by analytical saddlepoint computations which provide extremely accurate mid‐P‐values that are exact for most practical purposes and almost always more accurate than normal approximations. The speed of mid‐P‐value computation allows for the inversion of such tests to determine confidence intervals for the percentage increase in mean (or median) survival time per unit increase in dosage. The Canadian Journal of Statistics 37: 5‐16; 2009 © 2009 Statistical Society of Canada     

p-Value for two-Sample Linear Rank Tests Using Permutation Simulations and the Normal Approximation Method

Test statistics from the class of two-sample linear rank tests are commonly used to compare a treatment group with a control group. Two independent random samples of sizes m and n are drawn from two populations. As a result, N = m + n observations in total are obtained. The aim is to test the null hypothesis of identical distributions. The alternative hypothesis is that the populations are of the same form but with a different measure of central tendency. This article examines mid p-values from the null permutation distributions of tests based on the class of two-sample linear rank statistics. The results obtained indicate that normal approximation-based computations are very close to the permutation simulations, and they provide p-values that are close to the exact mid p-values for all practical purposes.     

Sunset Salvo

The Wilcoxon—Mann—Whitney test enjoys great popularity among scientists comparing two groups of observations, especially when measurements made on a continuous scale are non-normally distributed. Triggered by different results for the procedure from two statistics programs, we compared the outcomes from 11 PC-based statistics packages. The findings were that the delivered p values ranged from significant to nonsignificant at the 5% level, depending on whether a large-sample approximation or an exact permutation form of the test was used and, in the former case, whether or not a correction for continuity was used and whether or not a correction for ties was made. Some packages also produced pseudo-exact p values, based on the null distribution under the assumption of no ties. A further crucial point is that the variant of the algorithm used for computation by the packages is rarely indicated in the output or documented in the Help facility and the manuals. We conclude that the only accurate form of the Wilcoxon—Mann—Whitney procedure is one in which the exact permutation null distribution is compiled for the actual data.     

A new ranked set sampling protocol for the signed rank test

In this paper, we construct a new ranked set sampling protocol that maximizes the Pitman asymptotic efficiency of the signed rank test. The new sampling design is a function of the set size and independent order statistics. If the set size is odd and the underlying distribution is symmetric and unimodal, then the new sampling protocol quantifies only the middle observation. On the other hand, if the set size is even, the new sampling design quantifies the two middle observations. This data collection procedure for use in the signed rank test outperforms the data collection procedure in the standard ranked set sample. We show that the exact null distribution of the signed rank statistic W_RSS⁺ based on a data set generated by the new ranked set sample design for odd set sizes is the same as the null distribution of the simple random sample signed rank statistic W_SRS⁺ based on the same number of measured observations. For even set sizes, the exact null distribution of W_RSS⁺ is simulated.     

PERMUTATION TESTS AND OTHER TEST STATISTICS FOR ILL-BEHAVED DATA: EXPERIENCE OF THE NINDS t-PA STROKE TRIAL

Permutation tests are often used to analyze data since they may not require one to make assumptions regarding the form of the distribution to have a random and independent sample selection. We initially considered a permutation test to assess the treatment effect on computed tomography lesion volume in the National Institute of Neurological Disorders and Stroke (NINDS) t-PA Stroke Trial, which has highly skewed data. However, we encountered difficulties in summarizing the permutation test results on the lesion volume. In this paper, we discuss some aspects of permutation tests and illustrate our findings. This experience with the NINDS t-PA Stroke Trial data emphasizes that permutation tests are useful for clinical trials and can be used to validate assumptions of an observed test statistic. The permutation test places fewer restrictions on the underlying distribution but is not always distribution-free or an exact test, especially for ill-behaved data. Quasi-likelihood estimation using the generalized estimating equation (GEE) approach on transformed data seems to be a good choice for analyzing CT lesion data, based on both its corresponding permutation test and its clinical interpretation.     

Permutation Tests for Linear Models

Several approximate permutation tests have been proposed for tests of partial regression coefficients in a linear model based on sample partial correlations. This paper begins with an explanation and notation for an exact test. It then compares the distributions of the test statistics under the various permutation methods proposed, and shows that the partial correlations under permutation are asymptotically jointly normal with means 0 and variances 1. The method of Freedman & Lane (1983) is found to have asymptotic correlation 1 with the exact test, and the other methods are found to have smaller correlations with this test. Under local alternatives the critical values of all the approximate permutation tests converge to the same constant, so they all have the same asymptotic power. Simulations demonstrate these theoretical results.     

Comparing mean ranks for repeated measures data

Rank tests are considered that compare t treatments in repeated measures designs. A statistic is given that contains as special cases several that have been proposed for this problem, including one that corresponds to the randomized block ANOVA statistic applied to the rank transformed data. Another statistic is proposed, having a null distribution holding under more general conditions, that is the rank transform of the Hotelling statistic for repeated measures. A statistic of this type is also given for data that are ordered categorical rather than fully rankedo Unlike the Friedman statistic, the statistics discussed in this article utilize a single ranking of the entire sample. Power calculations for an underlying normal distribution indicate that the rank transformed ANOVA test can be substantially more powerful than the Friedman test.     

Distribution-free tests for cluster samples of ordinal responses

A simulation comparison is done of Mann–Whitney U test extensions recently proposed for simple cluster samples or for repeated ordinal responses. These are based on two approaches: the permutation approach of Fay and Gennings (four tests, two exact), and Edwardes’ approach (two asymptotic tests, one new). Edwardes’ approach permits confidence interval estimation, unlike the permutation approach. An appropriate parameter for estimation is P(X<Y)−P(X>Y), where X is the rank of a response from group 1 and Y is from group 2. The permutation tests are shown to be unsuitable for some survey data, since they are sensitive to a difference in cluster intra-correlations when there is no distribution difference between groups at the individual level. The exact permutation tests are of little use for less than seven clusters, precisely where they are most needed. Otherwise, the permutation tests perform well.     

Permutation techniques for analyzing multi-response data from randomized block experiments

A class of permutation techniques is presented for the randomized block design. This class is specifically devised for analyses involving multivariate data. A numerical example illustrates an application based on multivariate data. Many well known techniques are special cases of this class. Among these special cases are (i) the permutation version of the classical univariate technique for randomized blocks which 1s associated with analysis of variance, (ii) the Friedman randomized block test, (iii) one-sample matched-pair tests, (iv) the Pearson correlation measure, and (v) the Spearman rank correlation and foot-rule measures. Furthermore, variations and multivariate versions among this class suggest a variety of new techniques which have not received any previous attention.     

Smoothed alternatives of the two-sample median and Wilcoxon's rank sum tests

We discuss smoothed rank statistics for testing the location shift parameter of the two-sample problem. They are based on discrete test statistics – the median and Wilcoxon's rank sum tests. For the one-sample problem, Maesono et al. [Smoothed nonparametric tests and their properties. arXiv preprint. 2016; ArXiv:1610.02145] reported that some nonparametric discrete tests have a problem with their p-values because of their discreteness. The p-values of Wilcoxon's test are frequently smaller than those of the median test in the tail area. This leads to an arbitrary choice of the median and Wilcoxon's rank sum tests. To overcome this problem, we propose smoothed versions of those tests. The smoothed tests inherit the good properties of the original tests and are asymptotically equivalent to them. We study the significance probabilities and local asymptotic powers of the proposed tests.     

A Discussion of Permutation Tests Conditional to Observed Responses in Unreplicated 2 M Full Factorial Designs

ABSTRACT

In this article we present a new solution to test for effects in unreplicated two-level factorial designs. The proposed test statistic, in case the error components are normally distributed, follows an F random variable, though our attention is on its nonparametric permutation version. The proposed procedure does not require any transformation of data such as residualization and it is exact for each effect and distribution-free. Our main aim is to discuss a permutation solution conditional to the original vector of responses. We give two versions of the same nonparametric testing procedure in order to control both the individual error rate and the experiment-wise error rate. A power comparison with Loughin and Noble's test is provided in the case of a unreplicated 2⁴ full factorial design.