Comparison of nonparametric analysis of variance methods: A vote for van der Waerden

ABSTRACT

For two-way layouts in a between-subjects analysis of variance design, the parametric F-test is compared with seven nonparametric methods: rank transform (RT), inverse normal transform (INT), aligned rank transform (ART), a combination of ART and INT, Puri & Sen's L statistic, Van der Waerden, and Akritas and Brunners ANOVA-type statistics (ATS). The type I error rates and the power are computed for 16 normal and nonnormal distributions, with and without homogeneity of variances, for balanced and unbalanced designs as well as for several models including the null and the full model. The aim of this study is to identify a method that is applicable without too much testing for all the attributes of the plot. The Van der Waerden test shows the overall best performance though there are some situations in which it is disappointing. The Puri & Sen's and the ATS tests show generally very low power. These two and the other methods cannot keep the type I error rate under control in too many situations. Especially in the case of lognormal distributions, the use of any of the rank-based procedures can be dangerous for cell sizes above 10. As already shown by many other authors, nonnormal distributions do not violate the parametric F-test, but unequal variances do, and heterogeneity of variances leads to an inflated error rate more or less also for the nonparametric methods. Finally, it should be noted that some procedures show rising error rates with increasing cell sizes, the ART, especially for discrete variables, and the RT, Puri & Sen, and the ATS in the cases of heteroscedasticity.     

Tests for Symmetry Based on the One-Sample Wilcoxon Signed Rank Statistic

ABSTRACT

The one-sample Wilcoxon signed rank test was originally designed to test for a specified median, under the assumption that the distribution is symmetric, but it can also serve as a test for symmetry if the median is known. In this article we derive the Wilcoxon statistic as the first component of Pearson's X ² statistic for independence in a particularly constructed contingency table. The second and third components are new test statistics for symmetry. In the second part of the article, the Wilcoxon test is extended so that symmetry around the median and symmetry in the tails can be examined seperately. A trimming proportion is used to split the observations in the tails from those around the median. We further extend the method so that no arbitrary choice for the trimming proportion has to be made. Finally, the new tests are compared to other tests for symmetry in a simulation study. It is concluded that our tests often have substantially greater powers than most other tests.     

Continuity corrected approximations for and ‘exact’ inference with Pearson's X2

The classical adjustments for the inadequacy of the asymptotic distribution of Pearson's X² statistic, when some cells are sparse or the cell expectations are small, use continuity corrections and exact moments; the recent approach is to use computer based ‘exact inference’. In this paper we observe that the original exact test due to Freeman and Halton (Biometrika 38 (1951), 141–149) and its computer implementation are theoretically unsound. Furthermore, the corrected algorithmic version for the exact p-value in StatXact is practically useful in very few cases, and the results of its present version which includes Monte Carlo estimates can be highly variable. We then derive asymptotic expansions for the moments of the null distribution of Pearson's X², introduce a new method of correcting for discreteness and finite range of Pearson's X² as an alternative to the classical continuity correction, and use them to construct new and improved approximations for the null distribution. We also offer diagnostic criteria applicable to the tables for selecting an appropriate approximation. The exact methods and the competing approximations are studied and compared using thirteen test cases from the literature. It is concluded that the accuracy of the appropriate approximation is comparable with the truly exact method whenever it is available. The use of approximations is therefore preferable if the truly exact computer intensive solutions are unavailable or infeasible.     

Exact Statistical Tests for Heterogeneity of Frequencies Based on Extreme Values

Sophisticated statistical analyses of incidence frequencies are often required for various epidemiologic and biomedical applications. Among the most commonly applied methods is the Pearson's χ² test, which is structured to detect non specific anomalous patterns of frequencies and is useful for testing the significance for incidence heterogeneity. However, the Pearson's χ² test is not efficient for assessing the significance of frequency in a particular cell (or class) to be attributed to chance alone. We recently developed statistical tests for detecting temporal anomalies of disease cases based on maximum and minimum frequencies; these tests are actually designed to test of significance for a particular high or low frequency. The purpose of this article is to demonstrate merits of these tests in epidemiologic and biomedical studies. We show that our proposed methods are more sensitive and powerful for testing extreme cell counts than is the Pearson's χ² test. This feature could provide important and valuable information in epidemiologic or biomeidcal studies. We elucidated and illustrated the differences in sensitivity among our tests and the Pearson's χ² test by analyzing a data set of Langerhans cell histiocytosis cases and its hypothetical sets. We also computed and compared the statistical power of these methods using various sets of cell numbers and alternative frequencies. The investigation of statistical sensitivity and power presented in this work will provide investigators with useful guidelines for selecting the appropriate tests for their studies.     

A Discussion of Permutation Tests Conditional to Observed Responses in Unreplicated 2 M Full Factorial Designs

ABSTRACT

In this article we present a new solution to test for effects in unreplicated two-level factorial designs. The proposed test statistic, in case the error components are normally distributed, follows an F random variable, though our attention is on its nonparametric permutation version. The proposed procedure does not require any transformation of data such as residualization and it is exact for each effect and distribution-free. Our main aim is to discuss a permutation solution conditional to the original vector of responses. We give two versions of the same nonparametric testing procedure in order to control both the individual error rate and the experiment-wise error rate. A power comparison with Loughin and Noble's test is provided in the case of a unreplicated 2⁴ full factorial design.     

On Whiteman's and Storer's difference sets

In this paper, we prove that all Storer's difference sets are cyclic. We also prove that for p<1.8×10²⁵, Whiteman's difference sets exist if and only if (p,q)=(17,53) and (46817,140453).     

Smoothed alternatives of the two-sample median and Wilcoxon's rank sum tests

We discuss smoothed rank statistics for testing the location shift parameter of the two-sample problem. They are based on discrete test statistics – the median and Wilcoxon's rank sum tests. For the one-sample problem, Maesono et al. [Smoothed nonparametric tests and their properties. arXiv preprint. 2016; ArXiv:1610.02145] reported that some nonparametric discrete tests have a problem with their p-values because of their discreteness. The p-values of Wilcoxon's test are frequently smaller than those of the median test in the tail area. This leads to an arbitrary choice of the median and Wilcoxon's rank sum tests. To overcome this problem, we propose smoothed versions of those tests. The smoothed tests inherit the good properties of the original tests and are asymptotically equivalent to them. We study the significance probabilities and local asymptotic powers of the proposed tests.     

Mixed Graphical Model Selection Using Holm's Procedure

The problem of selecting a graphical model is considered as a performing simultaneously multiple tests. The control of the overall Type I error on the selected graph is done using the so famous Holm's procedure. We prove that when we use a consistent edge exclusion test the selected graph is asymptotically equal to the true graph with probability at least equal to a fixed level 1 ? α. This method is then used for the selection of mixed concentration graph models by performing the χ²-edge exclusion test. We also apply the method to two classical examples and to simulated data. We compare the overall error of the selected model with the one obtained using the stepwise method. We establish that the control is better when we use the Holm's procedure.     

On constructing T 2 control charts for retrospective examination

In this paper we consider the issue of constructing retrospective T ² control chart limits so as to control the overall probability of a false alarm at a specified value. We describe an exact method for constructing the control limits for retrospective examination. We then consider Bonferroni-adjustments to Alt's control limit and to the standard x ² control limit as alternatives to the exact limit since it is computationally cumbersome to find the exact limit. We present the results of some simulation experiments that are carried out to compare the performance of these control limits. The results indicate that the Bonferroni-adjusted Alt's control limit performs better that the Bonferroni-adjusted x ² control limit. Furthermore, it appears that the Bonferroni-adjusted Alt's control limit is more than adequate for controlling the overall false alarm probability at a specified value.     

A note on the multivariate multisample rank sum and median tests

The muitivariate nonparametric tests analogous to the univar-iate rank sum test and median test are contained in Puri and Sen (1970). These tests provided a practical alternative for the analysis of multivariate data when the assumptions of parametric methods are not satisfied.

In this paper maximum values for L_Nthe asymptotic chi-Square test statistic for both the Multivariate Multisample Rank Sum Test (MMRST) and the Multivariate Multisample Median Test (MMMT) are developed.     

Bayes A-optimal designs for comparing test treatments with a control

The problem of comparing v test treatments simultaneously with a control treatment when k, v ⩾ 3 is considered. Following the work of Majumdar (1992), we use exact design theory to derive Bayes A-optimal block designs and optimal Г-minimax designs for a more general prior assumption for the one-way elimination of heterogeneity model. Examples of robust optimal designs, highly efficient designs, and the comparisons of the approximate optimal designs that are derived by our methods and by some other existing rounding-off schemes when using Owen's procedure are also provided.     

A tool for systematically comparing the power of tests for normality

In this article, we describe a new approach to compare the power of different tests for normality. This approach provides the researcher with a practical tool for evaluating which test at their disposal is the most appropriate for their sampling problem. Using the Johnson systems of distribution, we estimate the power of a test for normality for any mean, variance, skewness, and kurtosis. Using this characterization and an innovative graphical representation, we validate our method by comparing three well-known tests for normality: the Pearson χ² test, the Kolmogorov–Smirnov test, and the D'Agostino–Pearson K ² test. We obtain such comparison for a broad range of skewness, kurtosis, and sample sizes. We demonstrate that the D'Agostino–Pearson test gives greater power than the others against most of the alternative distributions and at most sample sizes. We also find that the Pearson χ² test gives greater power than Kolmogorov–Smirnov against most of the alternative distributions for sample sizes between 18 and 330.     

Sensitivity of Rao's score test,the Wald test and the likelihood ratio test to nuisance parameters

The purpose of this paper is to compare the sensitivity of the likelihood ratio test, Rao's score test, and the Wald test to the change of the nuisance parameters. The main result is that, with an error of magnitude O(n⁻¹), the null distributions and the local alternative distributions of these tests are equally sensitive to nuisance parameter. We will also give accurate factorizations of these test statistics as quadratic forms, which are themselves useful for asymptotic analyses.     

Hayter and Tsui's test with double sampling for the vector mean of multivariate normal population

Abstract

In this paper, we introduce a version of Hayter and Tsui's statistical test with double sampling for the vector mean of a population under multivariate normal assumption. A study showed that this new test was more or as efficient than the well-known Hotelling's T² with double sampling. Some nice features of Hayter and Tsui's test are its simplicity of implementation and its capability of identifying the errant variables when the null hypothesis is rejected. Taking that into consideration, a new control chart called HTDS is also introduced as a tool to monitor multivariate process vector mean when using double sampling.     

Optimal locally robust M-estimates of regression

First, we show that many robust estimates of regression which depend only on the regression residuals (including M-, S-, Tau-, least median of squares-, least trimmed of squares- and some R-estimates) have infinite gross-error-sensitivity. More precisely, we show that the maximum-bias function of a large class of estimates, called residual admissible in Yohai and Zamar (Ann. Statist. 21, 1993, 1824–1842), is of order √ε near zero. Based on this finding we define a new robustness measure for estimates with B_T(ε) = o(ε^β), the contamination sensitivity of order β, which extends Hampel's gross error sensitivity for estimates with unbounded influence. We compute this measure for regression M-estimates with a general scale and show that β = 0.5 in this case. Then we solve a Hampel-like optimality problem, namely, one of minimizing the asymptotic variance subject to a bound on the contamination sensitivity of order β = 0.5, for estimates in this class. Finally, we show that a certain least α-quantile estimate has the smallest contamination sensitivity of order 0.5 among all residual admissible estimates. In the Gaussian case α = 0.683.     

Multivariate analysis of variance with additional data from an unknown subset of the populations

In this paper we consider the problem of testing the means of k multivariate normal populations with additional data from an unknown subset of the k populations. The purpose of this research is to offer test procedures utilizing all the available data for the multivariate analysis of variance problem because the additional data may contain valuable information about the parameters of the k populations. The standard procedure uses only the data from identified populations. We provide a test using all available data based upon Hotelling' s generalized T²statistic. The power of this test is computed using Betz's approximation of Hotelling' s generalized T²statistic by an F-distribution. A comparison of the power of the test and the standard test procedure is also given.     

A combined p-value test for multiple hypothesis testing

Tests that combine p-values, such as Fisher's product test, are popular to test the global null hypothesis H₀ that each of n component null hypotheses, H₁,…,H_n, is true versus the alternative that at least one of H₁,…,H_n is false, since they are more powerful than classical multiple tests such as the Bonferroni test and the Simes tests. Recent modifications of Fisher's product test, popular in the analysis of large scale genetic studies include the truncated product method (TPM) of Zaykin et al. (2002), the rank truncated product (RTP) test of Dudbridge and Koeleman (2003) and more recently, a permutation based test—the adaptive rank truncated product (ARTP) method of Yu et al. (2009). The TPM and RTP methods require users' specification of a truncation point. The ARTP method improves the performance of the RTP method by optimizing selection of the truncation point over a set of pre-specified candidate points. In this paper we extend the ARTP by proposing to use all the possible truncation points {1,…,n} as the candidate truncation points. Furthermore, we derive the theoretical probability distribution of the test statistic under the global null hypothesis H₀. Simulations are conducted to compare the performance of the proposed test with the Bonferroni test, the Simes test, the RTP test, and Fisher's product test. The simulation results show that the proposed test has higher power than the Bonferroni test and the Simes test, as well as the RTP method. It is also significantly more powerful than Fisher's product test when the number of truly false hypotheses is small relative to the total number of hypotheses, and has comparable power to Fisher's product test otherwise.     

The asymptotic distributions of maximum likelihood ratio test and maximally selected χ2-test in binomial observations

We considered binomial distributed random variables whose parameters are unknown and some of those parameters need to be estimated. We studied the maximum likelihood ratio test and the maximally selected χ²-test to detect if there is a change in the distributions among the random variables. Their limit distributions under the null hypothesis and their asymptotic distributions under the alternative hypothesis were obtained when the number of the observations is fixed. We discussed the properties of the limit distribution and found an efficient way to calculate the probability of multivariate normal random variables. Finally, those results for both tests have been applied to examples of Lindisfarne's data, the Talipes Data. Our conclusions are consistent with other researchers' findings.     

An Efficient Method to Generate Data and Compute Exact P-Values in Goodness-of-Fit Testing

In this article, we use a characterization of the set of sample counts that do not match with the null hypothesis of the test of goodness of fit. Two direct applications arise: first, to instantaneously generate data sets whose corresponding asymptotic P-values belong to a certain pre-defined range; and second, to compute exact P-values for this test in an efficient way. We present both issues before illustrating them by analyzing a couple of data sets. Method's efficiency is also assessed by means of simulations. We focus on Pearson's X ² statistic but the case of likelihood-ratio statistic is also discussed.     

Nonparametric predictive inference with right-censored data

We present a new method of statistical inference for data that include right-censored observations. The method is based on Hill's A_(n), a generalization of this assumption is introduced explicitly to deal with right-censored observations. We derive bounds for the predictive survival function. We compare this approach with Berliner and Hill's A_(n)-based method and with Kaplan and Meier's product-limit estimator. One of the attractive features of our method is detailed graphical presentation of the effects of right-censoring.