Review of Five Multilevel Analysis Programs: BMDP-5V,GENMOD, HLM,ML3, VARCL

A class of statistics is introduced for testing stochastic ordering between two independent distributions. This class includes as a special case the celebrated Mann—Whitney—Wilcoxon statistic. The new class is shown to be asymptotically normal both under the null and nonnull hypotheses. It is distribution-free. Using Pitman's asymptotic efficacy it is shown that for some alternatives the Mann—Whitney—Wilcoxon statistic is the member with the highest efficacy, although for others it is not, and the member with the highest efficacy is identified.     

Robustness without rank order statistics

An alternative to conventional rank tests based on a Euclidean distance analysis space is described. Comparisons based on exact probability values among classical two-sample t-tests and the Wilcoxon–Mann–Whitney test illustrate the advantages of the Euclidean distance analysis space alternative.     

A uniform saddlepoint expansion for the null‐distribution of the wilcoxon‐mann‐whitney statistic

The authors give the exact coefficient of 1/N in a saddlepoint approximation to the Wilcoxon‐Mann‐Whitney null‐distribution. This saddlepoint approximation is obtained from an Edgeworth approximation to the exponentially tilted distribution. Moreover, the rate of convergence of the relative error is uniformly of order O (1/N) in a large deviation interval as defined in Feller (1971). The proposed method for computing the coefficient of 1/N can be used to obtain the exact coefficients of 1/Nⁱ, for any i. The exact formulas for the cumulant generating function and the cumulants, needed for these results, are those of van Dantzig (1947‐1950).     

Notes on Kolassa’s Method for Estimating the Power of Wilcoxon–Mann–Whitney Test

The Kolassa method implemented in the nQuery Advisor software has been widely used for approximating the power of the Wilcoxon–Mann–Whitney (WMW) test for ordered categorical data, in which Edgeworth approximation is used to estimate the power of an unconditional test based on the WMW U statistic. When the sample size is small or when the sizes in the two groups are unequal, Kolassa’s method may yield quite poor approximation to the power of the conditional WMW test that is commonly implemented in statistical packages. Two modifications of Kolassa’s formula are proposed and assessed by simulation studies.     

Exact null distributions of quadratic distribution-free statistics for two-way classification

We present new techniques for computing exact distributions of ‘Friedman-type’ statistics. Representing the null distribution by a generating function allows for the use of general, not necessarily integer-valued rank scores. Moreover, we use symmetry properties of the multivariate generating function to accelerate computations. The methods also work for cases with ties and for permutation statistics. We discuss some applications: the classical Friedman rank test, the normal scores test, the Friedman permutation test, the Cochran–Cox test and the Kepner–Robinson test. Finally, we shortly discuss self-made software for computing exact p-values.     

An improved ranked set two‐sample mann‐whitney‐wilcoxon test

The authors present an improved ranked set two‐sample Mann‐Whitney‐Wilcoxon test for a location shift between samples from two distributions F and G. They define a function that measures the amount of information provided by each observation from the two samples, given the actual joint ranking of all the units in a set. This information function is used as a guide for improving the Pitman efficacy of the Mann‐Whitney‐Wilcoxon test. When the underlying distributions are symmetric, observations at their mode(s) must be quantified in order to gain efficiency. Analogous results are provided for asymmetric distributions.     

Implications of random cut‐points theory for the Mann‐Whitney and binomial tests

Through random cut‐points theory, the author extends inference for ordered categorical data to the unspecified continuum underlying the ordered categories. He shows that a random cut‐point Mann‐Whitney test yields slightly smaller p‐values than the conventional test for most data. However, when at least P% of the data lie in one of the k categories (with P = 80 for k = 2, P = 67 for k = 3,…, P = 18 for k = 30), he also shows that the conventional test can yield much smaller p‐values, and hence misleadingly liberal inference for the underlying continuum. The author derives formulas for exact tests; for k = 2, the Mann‐Whitney test is but a binomial test.     

WILCOXON-TYPE RANK-SUM PRECEDENCE TESTS

This paper introduces Wilcoxon‐type rank‐sum precedence tests for testing the hypothesis that two life‐time distribution functions are equal. They extend the precedence life‐test first proposed by Nelson in 1963. The paper proposes three Wilcoxon‐type rank‐sum precedence test statistics—the minimal, maximal and expected rank‐sum statistics—and derives their null distributions. Critical values are presented for some combinations of sample sizes, and the exact power function is derived under the Lehmann alternative. The paper examines the power properties of the Wilcoxon‐type rank‐sum precedence tests under a location‐shift alternative through Monte Carlo simulations, and it compares the power of the precedence test, the maximal precedence test and Wilcoxon rank‐sum test (based on complete samples). Two examples are presented for illustration.     

Critical Values for the Robust Rank-Order Test

ABSTRACT

The nonparametric Wilcoxon–Mann–Whitney test is commonly used by practitioners for detecting differences in location (mean, median) between two samples. Earlier work has shown this test to have a number of disadvantages, most of which are remedied by use of the alternative robust rank-order test. Use of the robust rank-order test has been limited, perhaps partly because exact critical values have up to now been available for only a small number of sample-size values, and not for all of the commonly used levels of significance. This article expands what is known about the distribution of the robust rank-order test statistic; critical values are given for more sample sizes and for more levels of significance.     

A similarity analysis of curves

The authors propose a method for comparing two samples of curves. The notion of similarity between two curves is the basis of three statistics they suggest for testing the null hypothesis of no difference between the two groups. They exploit standard tools from functional data analysis to preprocess the observed curves and use the permutation distribution under the null hypothesis to obtain p‐values for their tests. They explore the operating characteristics of these tests through simulations and as an application, compare the ganglioside distribution in brain tissue between old and young rats.     

Modified Wilcoxon–Mann–Whitney Test and Power Against Strong Null

The Wilcoxon–Mann–Whitney (WMW) test is a popular rank-based two-sample testing procedure for the strong null hypothesis that the two samples come from the same distribution. A modified WMW test, the Fligner–Policello (FP) test, has been proposed for comparing the medians of two populations. A fact that may be under-appreciated among some practitioners is that the FP test can also be used to test the strong null like the WMW. In this article, we compare the power of the WMW and FP tests for testing the strong null. Our results show that neither test is uniformly better than the other and that there can be substantial differences in power between the two choices. We propose a new, modified WMW test that combines the WMW and FP tests. Monte Carlo studies show that the combined test has good power compared to either the WMW and FP test. We provide a fast implementation of the proposed test in an open-source software. Supplementary materials for this article are available online.     

Foreword (and Preface)

We present new algorithms for computing the exact distributions and p-values of quadratic t-sample distribution-free statistics of Kruskal–Wallis type. These algorithms are presented in terms of generating functions. We show that our algorithm also works for cases with ties and that it is much faster than existing algorithms. Moreover, we show how to use the results for the Kruskal–Wallis type statistics to compute the exact null distribution of the Chacko–Shorack statistic.     

Exact distribution-free tests for equality of several linear models

Exact tests for the equality of several linear models are developed using permutation techniques. Two cases of the linear model, characterized by either stochastic or nonstochastic predictors, are considered: the linear regression model (LRM) and the general linear model (GLM). A general class of test statistics using the volume of simplexes as the basic unit of analysis is proposed for this problem. The resulting class of statistics is shown to be a natural generalization of the multi-response permutation procedure (MRPP) test statistics which have been shown to comprise many of the statistics used in both parametric and nonparametric analysis of the standard g—sample problem. In the LRM case, exact moments of all orders are derived for the permutation distribution of any test statistic in the general class. Moment-based approximation of significance levels is shown to be computationally feasible in the simple LRM.     

Distribution-free tests for cluster samples of ordinal responses

A simulation comparison is done of Mann–Whitney U test extensions recently proposed for simple cluster samples or for repeated ordinal responses. These are based on two approaches: the permutation approach of Fay and Gennings (four tests, two exact), and Edwardes’ approach (two asymptotic tests, one new). Edwardes’ approach permits confidence interval estimation, unlike the permutation approach. An appropriate parameter for estimation is P(X<Y)−P(X>Y), where X is the rank of a response from group 1 and Y is from group 2. The permutation tests are shown to be unsuitable for some survey data, since they are sensitive to a difference in cluster intra-correlations when there is no distribution difference between groups at the individual level. The exact permutation tests are of little use for less than seven clusters, precisely where they are most needed. Otherwise, the permutation tests perform well.     

A Nonparametric Test Procedure Based on a Group of Quantile Tests

In this article, we consider nonparametric test procedures based on a group of quantile test statistics. We consider the quadratic form for the two-sided test and the maximal and summing types of statistics for the one-sided alternatives. Then we derive the null limiting distributions of the proposed test statistics using the large sample approximation theory. Also, we consider applying the permutation principle to obtain the null distribution. In this vein, we may consider the supremum type, which should use the permutation principle for obtaining the null distribution. Then we illustrate our procedure with an example and compare the proposed tests with other existing tests including the individual quantile tests by obtaining empirical powers through simulation study. Also, we comment on the related discussions to this testing procedure as concluding remarks. Finally we prove the lemmas and theorems in the appendices.     

The performance of tests when observations have different variances

It is common to test if there is an effect due to a treatment. The commonly used tests have the assumption that the observations differ in location, and that their variances are the same over the groups. Different variances can arise if the observations being analyzed are means of different numbers of observations on individuals or slopes of growth curves with missing data. This study is concerned with cases in which the unequal variances are known, or known to a constant of proportionality. It examines the performance of the ttest, the Mann–Whitney–Wilcoxon Rank Sum test, the Median test, and the Van der Waerden test under these conditions. The t-test based on the weighted means is the likelihood ratio test under normality and has the usual optimality properties. The other tests are compared to it. One may align and scale the observations by subtracting the mean and dividing by the standard deviation of each point. This leads to other, analogous test statistics based on these adjusted observations. These statistics are also compared. Finally, the regression scores tests are compared to the other procedures.     

Log‐rank permutation tests for trend: saddlepoint p‐values and survival rate confidence intervals

Suppose p + 1 experimental groups correspond to increasing dose levels of a treatment and all groups are subject to right censoring. In such instances, permutation tests for trend can be performed based on statistics derived from the weighted log‐rank class. This article uses saddlepoint methods to determine the mid‐P‐values for such permutation tests for any test statistic in the weighted log‐rank class. Permutation simulations are replaced by analytical saddlepoint computations which provide extremely accurate mid‐P‐values that are exact for most practical purposes and almost always more accurate than normal approximations. The speed of mid‐P‐value computation allows for the inversion of such tests to determine confidence intervals for the percentage increase in mean (or median) survival time per unit increase in dosage. The Canadian Journal of Statistics 37: 5‐16; 2009 © 2009 Statistical Society of Canada     

Pairwise comparisons of treatments with ordinal responses in a two-way setting

In clinical studies, pairwise comparisons are frequently performed to examine differences in efficacy between treatments. The statistical methods of pairwise comparisons are available when treatment responses are measured on an ordinal scale. The Wilcoxon–Mann–Whitney test and the latent normal model are popular examples. However, these procedures cannot be used to compare treatments in parallel groups (a two-way design) when overall type I error must be controlled. In this paper, we explore statistical approaches to the pairwise testing of treatments that satisfy the requirements of a two-way layout. The results of our simulation indicate that the latent normal approach is superior to the Wilcoxon–Mann–Whitney test. Clinical examples are used to illustrate our suggested testing methods.     

Win statistics (win ratio,win odds,and net benefit) can complement one another to show the strength of the treatment effect on time-to-event outcomes

Conventional analyses of a composite of multiple time-to-event outcomes use the time to the first event. However, the first event may not be the most important outcome. To address this limitation, generalized pairwise comparisons and win statistics (win ratio, win odds, and net benefit) have become popular and have been applied to clinical trial practice. However, win ratio, win odds, and net benefit have typically been used separately. In this article, we examine the use of these three win statistics jointly for time-to-event outcomes. First, we explain the relation of point estimates and variances among the three win statistics, and the relation between the net benefit and the Mann–Whitney U statistic. Then we explain that the three win statistics are based on the same win proportions, and they test the same null hypothesis of equal win probabilities in two groups. We show theoretically that the Z-values of the corresponding statistical tests are approximately equal; therefore, the three win statistics provide very similar p-values and statistical powers. Finally, using simulation studies and data from a clinical trial, we demonstrate that, when there is no (or little) censoring, the three win statistics can complement one another to show the strength of the treatment effect. However, when the amount of censoring is not small, and without adjustment for censoring, the win odds and the net benefit may have an advantage for interpreting the treatment effect; with adjustment (e.g., IPCW adjustment) for censoring, the three win statistics can complement one another to show the strength of the treatment effect. For calculations we use the R package WINS, available on the CRAN (Comprehensive R Archive Network).     

Robust inference from multiple test statistics via permutations: a better alternative to the single test statistic approach for randomized trials

Formal inference in randomized clinical trials is based on controlling the type I error rate associated with a single pre‐specified statistic. The deficiency of using just one method of analysis is that it depends on assumptions that may not be met. For robust inference, we propose pre‐specifying multiple test statistics and relying on the minimum p‐value for testing the null hypothesis of no treatment effect. The null hypothesis associated with the various test statistics is that the treatment groups are indistinguishable. The critical value for hypothesis testing comes from permutation distributions. Rejection of the null hypothesis when the smallest p‐value is less than the critical value controls the type I error rate at its designated value. Even if one of the candidate test statistics has low power, the adverse effect on the power of the minimum p‐value statistic is not much. Its use is illustrated with examples. We conclude that it is better to rely on the minimum p‐value rather than a single statistic particularly when that single statistic is the logrank test, because of the cost and complexity of many survival trials. Copyright © 2013 John Wiley & Sons, Ltd.