A comparative study of tests for paired lifetime data

In this paper, we investigate different procedures for testing the equality of two mean survival times in paired lifetime studies. We consider Owen’s M-test and Q-test, a likelihood ratio test, the paired t-test, the Wilcoxon signed rank test and a permutation test based on log-transformed survival times in the comparative study. We also consider the paired t-test, the Wilcoxon signed rank test and a permutation test based on original survival times for the sake of comparison. The size and power characteristics of these tests are studied by means of Monte Carlo simulations under a frailty Weibull model. For less skewed marginal distributions, the Wilcoxon signed rank test based on original survival times is found to be desirable. Otherwise, the M-test and the likelihood ratio test are the best choices in terms of power. In general, one can choose a test procedure based on information about the correlation between the two survival times and the skewness of the marginal survival distributions.     

The split sample permutation t-tests

Without the exchangeability assumption, permutation tests for comparing two population means do not provide exact control of the probability of making a Type I error. Another drawback of permutation tests is that it cannot be used to test hypothesis about one population. In this paper, we propose a new type of permutation tests for testing the difference between two population means: the split sample permutation t-tests. We show that the split sample permutation t-tests do not require the exchangeability assumption, are asymptotically exact and can be easily extended to testing hypothesis about one population. Extensive simulations were carried out to evaluate the performance of two specific split sample permutation t-tests: the split in the middle permutation t-test and the split in the end permutation t-test. The simulation results show that the split in the middle permutation t-test has comparable performance to the permutation test if the population distributions are symmetric and satisfy the exchangeability assumption. Otherwise, the split in the end permutation t-test has significantly more accurate control of level of significance than the split in the middle permutation t-test and other existing permutation tests.     

Nonparametric test for the homogeneity of the overall variability

In this paper, we propose a nonparametric test for homogeneity of overall variabilities for two multi-dimensional populations. Comparisons between the proposed nonparametric procedure and the asymptotic parametric procedure and a permutation test based on standardized generalized variances are made when the underlying populations are multivariate normal. We also study the performance of these test procedures when the underlying populations are non-normal. We observe that the nonparametric procedure and the permutation test based on standardized generalized variances are not as powerful as the asymptotic parametric test under normality. However, they are reliable and powerful tests for comparing overall variability under other multivariate distributions such as the multivariate Cauchy, the multivariate Pareto and the multivariate exponential distributions, even with small sample sizes. A Monte Carlo simulation study is used to evaluate the performance of the proposed procedures. An example from an educational study is used to illustrate the proposed nonparametric test.     

A non-parametric maximum test for the Behrens–Fisher problem

Non-normality and heteroscedasticity are common in applications. For the comparison of two samples in the non-parametric Behrens–Fisher problem, different tests have been proposed, but no single test can be recommended for all situations. Here, we propose combining two tests, the Welch t test based on ranks and the Brunner–Munzel test, within a maximum test. Simulation studies indicate that this maximum test, performed as a permutation test, controls the type I error rate and stabilizes the power. That is, it has good power characteristics for a variety of distributions, and also for unbalanced sample sizes. Compared to the single tests, the maximum test shows acceptable type I error control.     

More powerful permutation test based on multistage ranked set sampling

Many researches have used ranked set sampling (RSS) method instead of simple random sampling (SRS) to improve power of some nonparametric tests. In this study, the two-sample permutation test within multistage ranked set sampling (MSRSS) is proposed and investigated. The power of this test is compared with the SRS permutation test for some symmetric and asymmetric distributions through Monte Carlo simulations. It has been found that this test is more powerful than the SRS permutation test; its power increased by set size and/or number of cycles and/or number of stages. Symmetric distributions power increased better than asymmetric distributions power.     

Exact permutation inference for two sample repeated measures data

Experiments in which very few units are measured many times sometimes present particular difficulties. Interest often centers on simple location shifts between two treatment groups, but appropriate modeling of the error distribution can be challenging. For example, normality may be difficult to verify, or a single transformation stabilizing variance or improving normality for all units and all measurements may not exist. We propose an analysis of two sample repeated measures data based on the permutation distribution of units. This provides a distribution free alternative to standard analyses. The analysis includes testing, estimation and confidence intervals. By assuming a certain structure in the location shift model, the dimension of the problem is reduced by analyzing linear combinations of the marginal statistics. Recently proposed algorithms for computation of two sample permutation distributions, require only a few seconds for experiments having as many as 100 units and any number of repeated measures. The test has high asymptotic efficiency and good power with respect to tests based on the normal distribution. Since the computational burden is minimal, approximation of the permutation distribution is unnecessary.     

Exact null distributions of quadratic distribution-free statistics for two-way classification

We present new techniques for computing exact distributions of ‘Friedman-type’ statistics. Representing the null distribution by a generating function allows for the use of general, not necessarily integer-valued rank scores. Moreover, we use symmetry properties of the multivariate generating function to accelerate computations. The methods also work for cases with ties and for permutation statistics. We discuss some applications: the classical Friedman rank test, the normal scores test, the Friedman permutation test, the Cochran–Cox test and the Kepner–Robinson test. Finally, we shortly discuss self-made software for computing exact p-values.     

Permutation Tests for Linear Models

Several approximate permutation tests have been proposed for tests of partial regression coefficients in a linear model based on sample partial correlations. This paper begins with an explanation and notation for an exact test. It then compares the distributions of the test statistics under the various permutation methods proposed, and shows that the partial correlations under permutation are asymptotically jointly normal with means 0 and variances 1. The method of Freedman & Lane (1983) is found to have asymptotic correlation 1 with the exact test, and the other methods are found to have smaller correlations with this test. Under local alternatives the critical values of all the approximate permutation tests converge to the same constant, so they all have the same asymptotic power. Simulations demonstrate these theoretical results.     

A Discussion of Permutation Tests Conditional to Observed Responses in Unreplicated 2 M Full Factorial Designs

ABSTRACT

In this article we present a new solution to test for effects in unreplicated two-level factorial designs. The proposed test statistic, in case the error components are normally distributed, follows an F random variable, though our attention is on its nonparametric permutation version. The proposed procedure does not require any transformation of data such as residualization and it is exact for each effect and distribution-free. Our main aim is to discuss a permutation solution conditional to the original vector of responses. We give two versions of the same nonparametric testing procedure in order to control both the individual error rate and the experiment-wise error rate. A power comparison with Loughin and Noble's test is provided in the case of a unreplicated 2⁴ full factorial design.     

A general theory of hypothesis testing based on rankings

In nonparametric statistics, a hypothesis testing problem based on the ranks of the data gives rise to two separate permutation sets corresponding to the null and to the alternative hypothesis, respectively. A modification of Critchlow's unified approach to hypothesis testing is proposed. By defining the distance between permutation sets to be the average distance between pairs of permutations, one from each set, various test statistics are derived for the multi-sample location problem and the two-way layout. The asymptotic distributions of the test statistics are computed under both the null and alternative hypotheses. Some comparisons are made on the basis of the asymptotic relative efficiency.     

Permutation tests for homogeneity based on some characterizations

In this work, non parametric tests are proposed for testing the homogeneity of two or more populations. The tests are based on recently obtained characterizations. The test procedure is based on the permutation bootstrap technique. For the two-sample case the new tests are compared with permutation tests based on the empirical characteristic function and some other tests. The comparison is fulfilled via a Monte Carlo simulation.     

Comparing Two Survival Time Distributions: An Investigation of Several Weight Functions for the Weighted Logrank Statistic

In survival analysis, it is often of interest to test whether or not two survival time distributions are equal, specifically in the presence of censored data. One very popular test statistic utilized in this testing procedure is the weighted logrank statistic. Much attention has been focused on finding flexible weight functions to use within the weighted logrank statistic, and we propose yet another. We demonstrate our weight function to be more stable than one of the most popular, which is given by Fleming and Harrington, by means of asymptotic normal tests, bootstrap tests and permutation tests performed on two datasets with a variety of characteristics.     

Properties of a randomization test for multifactor comparisons of groups

Multivariate hypothesis testing in studies of vegetation is likely to be hindered by unrealistic assumptions when based on conventional statistical methods. This can be overcome by randomization tests. In this paper, the accuracy and power of a MANOVA randomization test are evaluated for one and two factors with interaction with simulated data from three distributions. The randomization test is based on the partitioning of sum of squares computed from Euclidean distances. In one-factor designs, sample size and variance inequality were evaluated. The results showed a high level of accuracy. The power curve was higher with normal distribution, lower with uniform, intermediate with lognormal and was sensitive to variance inequality. In two-factor designs, three methods of permutations and two statistics were compared. The results showed that permutation of the residuals with F _pseudo is accurate and can give good power for testing the interaction and restricted permutation for testing main factors.     

A Class of Permutation Tests for the Equality of Two Marginal Survival Functions Using Paired Censored Data

We studied several test statistics for testing the equality of marginal survival functions of paired censored data. The null distribution of the test statistics was approximated by permutation. These tests do not require explicit modeling or estimation of the within-pair correlation, accommodate both paired data and singletons, and the computation is straightforward with most statistical software. Numerical studies showed that these tests have competitive size and power performance. One test statistic has higher power than previously published test statistics when the two survival functions under comparison cross. We illustrate use of these tests in a propensity score matched dataset.     

Book Reviews

The Levene test is a widely used test for detecting differences in dispersion. The modified Levene transformation using sample medians is considered in this article. After Levene's transformation the data are not normally distributed, hence, nonparametric tests may be useful. As the Wilcoxon rank sum test applied to the transformed data cannot control the type I error rate for asymmetric distributions, a permutation test based on reallocations of the original observations rather than the absolute deviations was investigated. Levene's transformation is then only an intermediate step to compute the test statistic. Such a Levene test, however, cannot control the type I error rate when the Wilcoxon statistic is used; with the Fisher–Pitman permutation test it can be extremely conservative. The Fisher–Pitman test based on reallocations of the transformed data seems to be the only acceptable nonparametric test. Simulation results indicate that this test is on average more powerful than applying the t test after Levene's transformation, even when the t test is improved by the deletion of structural zeros.     

A Permutation Based Procedure for Classification Assessment

This article proposes a permutation procedure for evaluating the performance of different classification methods. In particular, we focus on two of the most widespread and used classification methodologies: latent class analysis and k-means clustering. The classification performance is assessed by means of a permutation procedure which allows for a direct comparison of the methods, the development of a statistical test, and points out better potential solutions. Our proposal provides an innovative framework for the validation of the data partitioning and offers a guide in the choice of which classification procedure should be used     

Distribution-free tests for cluster samples of ordinal responses

A simulation comparison is done of Mann–Whitney U test extensions recently proposed for simple cluster samples or for repeated ordinal responses. These are based on two approaches: the permutation approach of Fay and Gennings (four tests, two exact), and Edwardes’ approach (two asymptotic tests, one new). Edwardes’ approach permits confidence interval estimation, unlike the permutation approach. An appropriate parameter for estimation is P(X<Y)−P(X>Y), where X is the rank of a response from group 1 and Y is from group 2. The permutation tests are shown to be unsuitable for some survey data, since they are sensitive to a difference in cluster intra-correlations when there is no distribution difference between groups at the individual level. The exact permutation tests are of little use for less than seven clusters, precisely where they are most needed. Otherwise, the permutation tests perform well.     

Saddlepoint p-values for a class of tests for comparing competing risks with censored data

One of the general problems in clinical trials and mortality rates is the comparison of competing risks. Most of the test statistics used for independent and dependent risks with censored data belong to the class of weighted linear rank tests in its multivariate version. In this paper, we introduce the saddlepoint approximations as accurate and fast approximations for the exact p-values of this class of tests instead of the asymptotic and permutation simulated calculations. Real data examples and extensive simulation studies showed the accuracy and stability performance of the saddlepoint approximations over different scenarios of lifetime distributions, sample sizes and censoring.     

Conditional Studentized Survival Tests for Randomly Censored Models

It is shown that in the case of heterogenous censoring distributions Studentized survival tests can be carried out as conditional permutation tests given the order statistics and their censoring status. The result is based on a conditional central limit theorem for permutation statistics. It holds for linear test statistics as well as for sup-statistics. The procedure works under one of the following general circumstances for the two-sample problem: the unbalanced sample size case, highly censored data, certain non-convergent weight functions or under alternatives. For instance, the two-sample log rank test can be carried out asymptotically as a conditional test if the relative amount of uncensored observations vanishes asymptotically as long as the number of uncensored observations becomes infinite. Similar results hold whenever the sample sizes and are unbalanced in the sense that and hold.     

A combined test for differences in scale based on the interquantile range

A class of tests due to Shoemaker (Commun Stat Simul Comput 28: 189–205, 1999) for differences in scale which is valid for a variety of both skewed and symmetric distributions when location is known or unknown is considered. The class is based on the interquantile range and requires that the population variances are finite. In this paper, we firstly propose a permutation version of it that does not require the condition of finite variances and is remarkably more powerful than the original one. Secondly we solve the question of what quantile choose by proposing a combined interquantile test based on our permutation version of Shoemaker tests. Shoemaker showed that the more extreme interquantile range tests are more powerful than the less extreme ones, unless the underlying distributions are very highly skewed. Since in practice you may not know if the underlying distributions are very highly skewed or not, the question arises. The combined interquantile test solves this question, is robust and more powerful than the stand alone tests. Thirdly we conducted a much more detailed simulation study than that of Shoemaker (1999) that compared his tests to the F and the squared rank tests showing that his tests are better. Since the F and the squared rank test are not good for differences in scale, his results suffer of such a drawback, and for this reason instead of considering the squared rank test we consider, following the suggestions of several authors, tests due to Brown–Forsythe (J Am Stat Assoc 69:364–367, 1974), Pan (J Stat Comput Simul 63:59–71, 1999), O’Brien (J Am Stat Assoc 74:877–880, 1979) and Conover et al. (Technometrics 23:351–361, 1981).