Nonparametric tests for the mean of a non-negative population

We construct level-α tests for testing the null hypothesis that the mean of a non-negative population falls below a prespecified nominal value. These tests make no assumption about the distribution function other than that it be supported on [0,∞). Simple tests are derived based on either the sample mean or the sample product. The nonparametric likelihood ratio test is also discussed in this context. We also derive the uniformly most powerful monotone (UMP) tests for special cases.     

Wilcoxon's signed‐rank statistic: what null hypothesis and why it matters

In statistical literature, the term ‘signed‐rank test’ (or ‘Wilcoxon signed‐rank test’) has been used to refer to two distinct tests: a test for symmetry of distribution and a test for the median of a symmetric distribution, sharing a common test statistic. To avoid potential ambiguity, we propose to refer to those two tests by different names, as ‘test for symmetry based on signed‐rank statistic’ and ‘test for median based on signed‐rank statistic’, respectively. The utility of such terminological differentiation should become evident through our discussion of how those tests connect and contrast with sign test and one‐sample t‐test. Published 2014. This article is a U.S. Government work and is in the public domain in the USA.     

Two separate effects of variance heterogeneity on the validity and power of significance tests of location

Heterogeneity of variances of treatment groups influences the validity and power of significance tests of location in two distinct ways. First, if sample sizes are unequal, the Type I error rate and power are depressed if a larger variance is associated with a larger sample size, and elevated if a larger variance is associated with a smaller sample size. This well-established effect, which occurs in t and F tests, and to a lesser degree in nonparametric rank tests, results from unequal contributions of pooled estimates of error variance in the computation of test statistics. It is observed in samples from normal distributions, as well as non-normal distributions of various shapes. Second, transformation of scores from skewed distributions with unequal variances to ranks produces differences in the means of the ranks assigned to the respective groups, even if the means of the initial groups are equal, and a subsequent inflation of Type I error rates and power. This effect occurs for all sample sizes, equal and unequal. For the t test, the discrepancy diminishes, and for the Wilcoxon–Mann–Whitney test, it becomes larger, as sample size increases. The Welch separate-variance t test overcomes the first effect but not the second. Because of interaction of these separate effects, the validity and power of both parametric and nonparametric tests performed on samples of any size from unknown distributions with possibly unequal variances can be distorted in unpredictable ways.     

On the sample breakdown robustness of some nonparametric tests

In this paper, relying on the sample breakdown points, we investigate the sample breakdown properties of some nonparametric tests. It is shown that the sample breakdown points of the sign test asymptotically dominate those of the Wilcoxon test for one–sided hypotheses, However, the different conclusion is derived in the case of testing some shrinking neighborhood hypotheses. The breakdown behaviors of the Kolmogorov test and X²–test are also explored. These studies unify or refine some existing breakdown analyses of tests.     

A powerful and interpretable alternative to the Jarque–Bera test of normality based on 2nd-power skewness and kurtosis,using the Rao's score test on the APD family

We introduce the 2nd-power skewness and kurtosis, which are interesting alternatives to the classical Pearson's skewness and kurtosis, called 3rd-power skewness and 4th-power kurtosis in our terminology. We use the sample 2nd-power skewness and kurtosis to build a powerful test of normality. This test can also be derived as Rao's score test on the asymmetric power distribution, which combines the large range of exponential tail behavior provided by the exponential power distribution family with various levels of asymmetry. We find that our test statistic is asymptotically chi-squared distributed. We also propose a modified test statistic, for which we show numerically that the distribution can be approximated for finite sample sizes with very high precision by a chi-square. Similarly, we propose a directional test based on sample 2nd-power kurtosis only, for the situations where the true distribution is known to be symmetric. Our tests are very similar in spirit to the famous Jarque–Bera test, and as such are also locally optimal. They offer the same nice interpretation, with in addition the gold standard power of the regression and correlation tests. An extensive empirical power analysis is performed, which shows that our tests are among the most powerful normality tests. Our test is implemented in an R package called PoweR.     

An Empirical Analysis of Some Nonparametric Goodness-of-Fit Tests for Censored Data

In this article, we consider some nonparametric goodness-of-fit tests for right censored samples, viz., the modified Kolmogorov, Cramer–von Mises–Smirnov, Anderson–Darling, and Nikulin–Rao–Robson χ² tests. We also consider an approach based on a transformation of the original censored sample to a complete one and the subsequent application of classical goodness-of-fit tests to the pseudo-complete sample. We then compare these tests in terms of power in the case of Type II censored data along with the power of the Neyman–Pearson test, and draw some conclusions. Finally, we present an illustrative example.     

Sample size determination for the weighted log‐rank test with the Fleming–Harrington class of weights in cancer vaccine studies

In recent years, immunological science has evolved, and cancer vaccines are available for treating existing cancers. Because cancer vaccines require time to elicit an immune response, a delayed treatment effect is expected. Accordingly, the use of weighted log‐rank tests with the Fleming–Harrington class of weights is proposed for evaluation of survival endpoints. We present a method for calculating the sample size under assumption of a piecewise exponential distribution for the cancer vaccine group and an exponential distribution for the placebo group as the survival model. The impact of delayed effect timing on both the choice of the Fleming–Harrington's weights and the increment in the required number of events is discussed. Copyright © 2014 John Wiley & Sons, Ltd.     

Shapiro–Francia test compared to other normality test using expected p-value

The Shapiro–Francia (SF) normality test is an important test in statistical modelling. However, little has been done by researchers to compare the performance of this test to other normality tests. This paper therefore measures the performance of the SF and other normality tests by studying the distribution of their p-values. For the purpose of this study, we selected eight well-known normality tests to compare with the SF test: (i) Kolmogorov–Smirnov (KS), (ii) Anderson–Darling (AD), (iii) Cramer von Mises (CM), (iv) Lilliefors (LF), (v) Shapiro–Wilk (SW), (vi) Pearson chi-square (PC), (vii) Jarque– Bera (JB) and (viii) D'Agostino (DA). The distribution of p-values of these normality tests were obtained by generating data from normal distribution and well-known symmetric non-normal distribution at various sample sizes (small, medium and large). Our simulation results showed that the SF normality test was the best test statistic in detecting deviation from normality among the nine tests considered at all sample sizes.     

Nonparametric two-sample tests for dispersion differences based on placements

Two different two-sample tests for dispersion differences based on placement statistics are proposed. The means and variances of the test statistics are derived, and asymptotic normality is established for both. Variants of the proposed tests based on reversing the X and Y labels in the test statistic calculations are shown to have different small-sample properties; for both pairs of tests, one member of the pair will be resolving, the other nonresolving. The proposed tests are similar in spirit to the dispersion tests of both Mood and Hollander; comparative simulation results for these four tests are given. For small sample sizes, the powers of the proposed tests are approximately equal to the powers of the tests of both Mood and Hollander for samples from the normal, Cauchy and exponential distributions. The one-sample limiting distributions are also provided, yielding useful approximations to the exact tests when one sample is much larger than the other. A bootstrap test may alternatively be performed. The proposed test statistics may be used with lightly censored data by substituting Kaplan-Meier estimates for the empirical distribution functions.     

Small sample power of multivariate nonparametric tests for monotone trend

Bhattacharyya and Kioiz (1966) propose two multivariate nonparametric tests for monotone trend, one involving coordinate-wise Mann statistics and the other, coordinate-wise Spearman statistics. Dietz and Killeen (1981) propose a different test statistic based on coordinate-wise Mann statistics. The Pitman asymptotic relative efficiency of all three tests with respect to a normal theory competitor equals the cube root of the efficiency of a multivariate signed rank test with respect to Hotelling's T2. In this article, the small sample power of the nonparametric tests, the normal theory test, and a Bonferroni approach involving coordinate-wise univariate Mann or Spearman tests is examined in a simulation study. The Mann statistic of Dietz and Killeen and the Spearman statistic of Bhattacharyya and Klotz are found to perform well under both null and alternative hypotheses     

A monte carlo study of the wald,lm, and lr tests in a heteroscedastic linear model

In this paper, we examine by Monte Carlo experiments the small sample properties of the W (Wald), LM (Lagrange Multiplier) and LR (Likelihood Ratio) tests for equality between sets of coefficients in two linear regressions under heteroscedasticity. The small sample properties of the size-corrected W, LM and LR tests proposed by Rothenberg (1984) are also examined and it is shown that the performances of the size-corrected W and LM tests are very good. Further, we examine the two-stage test which consists of a test for homoscedasticity followed by the Chow (1960) test if homoscedasticity is indicated or one of the W, LM or LR tests if heteroscedasticity should be assumed. It is shown that the pretest does not reduce much the bias in the size when the sizecorrected citical values are used in the W, LM and LR tests.     

Multivariate two‐sample permutation tests for trials with multiple time‐to‐event outcomes

Clinical trials involving multiple time‐to‐event outcomes are increasingly common. In this paper, permutation tests for testing for group differences in multivariate time‐to‐event data are proposed. Unlike other two‐sample tests for multivariate survival data, the proposed tests attain the nominal type I error rate. A simulation study shows that the proposed tests outperform their competitors when the degree of censored observations is sufficiently high. When the degree of censoring is low, it is seen that naive tests such as Hotelling's T² outperform tests tailored to survival data. Computational and practical aspects of the proposed tests are discussed, and their use is illustrated by analyses of three publicly available datasets. Implementations of the proposed tests are available in an accompanying R package.     

A comparison of several adaptive tests for paired data

In the last few years, two adaptive tests for paired data have been proposed. One test proposed by Freidlin et al. [On the use of the Shapiro–Wilk test in two-stage adaptive inference for paired data from moderate to very heavy tailed distributions, Biom. J. 45 (2003), pp. 887–900] is a two-stage procedure that uses a selection statistic to determine which of three rank scores to use in the computation of the test statistic. Another statistic, proposed by O'Gorman [Applied Adaptive Statistical Methods: Tests of Significance and Confidence Intervals, Society for Industrial and Applied Mathematics, Philadelphia, 2004], uses a weighted t-test with the weights determined by the data. These two methods, and an earlier rank-based adaptive test proposed by Randles and Hogg [Adaptive Distribution-free Tests, Commun. Stat. 2 (1973), pp. 337–356], are compared with the t-test and to Wilcoxon's signed-rank test. For sample sizes between 15 and 50, the results show that the adaptive test proposed by Freidlin et al. and the adaptive test proposed by O'Gorman have higher power than the other tests over a range of moderate to long-tailed symmetric distributions. The results also show that the test proposed by O'Gorman has greater power than the other tests for short-tailed distributions. For sample sizes greater than 50 and for small sample sizes the adaptive test proposed by O'Gorman has the highest power for most distributions.     

Small sample equivalence tests for exponentiality

We consider small sample equivalence tests for exponentialy. Statistical inference in this setting is particularly challenging since equivalence testing procedures typically require much larger sample sizes, in comparison with classical “difference tests,” to perform well. We make use of Butler's marginal likelihood for the shape parameter of a gamma distribution in our development of small sample equivalence tests for exponentiality. We consider two procedures using the principle of confidence interval inclusion, four Bayesian methods, and the uniformly most powerful unbiased (UMPU) test where a saddlepoint approximation to the intractable distribution of a canonical sufficient statistic is used. We perform small sample simulation studies to assess the bias of our various tests and show that all of the Bayes posteriors we consider are integrable. Our simulation studies show that the saddlepoint-approximated UMPU method performs remarkably well for small sample sizes and is the only method that consistently exhibits an empirical significance level close to the nominal 5% level.     

Goodness-of-fit tests for location–scale families based on random distance

For location–scale families, we consider a random distance between the sample order statistics and the quasi sample order statistics derived from the null distribution as a measure of discrepancy. The conditional qth quantile and expectation of the random discrepancy on the given sample are chosen as test statistics. Simulation results of powers against various alternatives are illustrated under the normal and exponential hypotheses for moderate sample size. The proposed tests, especially the qth quantile tests with a small or large q, are shown to be more powerful than other prominent goodness-of-fit tests in most cases.     

On the power of the t-test and some rank tests when outliers may be present

The power of some rank tests, used for testing the hypothesis of shift, is found when the underlying distributions contain outliers. The outliers are assumed to occur as the result of mixing two normal distributions with common variance. A small sample case shows how the scores for the rank tests are found and the exact power is computed for each of these rank tests. A Monte Carlo study provides an estimate of the power of the usual two sample t-test.     

Power transformations to induce normality and their applications

Summary.  Random variables which are positive linear combinations of positive independent random variables can have heavily right-skewed finite sample distributions even though they might be asymptotically normally distributed. We provide a simple method of determining an appropriate power transformation to improve the normal approximation in small samples. Our method contains the Wilson–Hilferty cube root transformation for χ ² random variables as a special case. We also provide some important examples, including test statistics of goodness-of-fit and tail index estimators, where such power transformations can be applied. In particular, we study the small sample behaviour of two goodness-of-fit tests for time series models which have been proposed recently in the literature. Both tests are generalizations of the popular Box–Ljung–Pierce portmanteau test, one in the time domain and the other in the frequency domain. A power transformation with a finite sample mean and variance correction is proposed, which ameliorates the small sample effect. It is found that the corrected versions of the tests have markedly better size properties. The correction is also found to result in an overall increase in power which can be significant under certain alternatives. Furthermore, the corrected tests also have better power than the Box–Ljung–Pierce portmanteau test, unlike the uncorrected versions.     

Sample size calculation for the one‐sample log‐rank test

In this paper, an exact variance of the one‐sample log‐rank test statistic is derived under the alternative hypothesis, and a sample size formula is proposed based on the derived exact variance. Simulation results showed that the proposed sample size formula provides adequate power to design a study to compare the survival of a single sample with that of a standard population. Copyright © 2014 John Wiley & Sons, Ltd.     

A Decomposition of Pearson–Fisher and Dzhaparidze–Nikulin Statistics and Some Ideas for a More Powerful Test Construction

An explicit decomposition on asymptotically independent distributed as chi-squared with one degree of freedom components of the Pearson–Fisher and Dzhaparidze–Nikulin tests is presented. The decomposition is formally the same for both tests and is valid for any partitioning of a sample space. Vector-valued tests, components of which can be not only different scalar tests based on the same sample, but also scalar tests based on components or groups of components of the same statistic are considered. Numerical examples illustrating the idea are presented.     

On the Exact Size of Tests of Treatment Effects in Multi‐Arm Clinical Trials

When testing treatment effects in multi‐arm clinical trials, the Bonferroni method or the method of Simes 1986) is used to adjust for the multiple comparisons. When control of the family‐wise error rate is required, these methods are combined with the close testing principle of Marcus et al. (1976). Under weak assumptions, the resulting p‐values all give rise to valid tests provided that the basic test used for each treatment is valid. However, standard tests can be far from valid, especially when the endpoint is binary and when sample sizes are unbalanced, as is common in multi‐arm clinical trials. This paper looks at the relationship between size deviations of the component test and size deviations of the multiple comparison test. The conclusion is that multiple comparison tests are as imperfect as the basic tests at nominal size α/m where m is the number of treatments. This, admittedly not unexpected, conclusion implies that these methods should only be used when the component test is very accurate at small nominal sizes. For binary end‐points, this suggests use of the parametric bootstrap test. All these conclusions are supported by a detailed numerical study.