Size performance of some tests in one-way anova

Tsui and Weerahandi (1989) introduced the notion of generalized p-values and since then this idea is used to solve many statistical testing problems. Heteroskedasticity is one of the major practical problems encountered in ANOVA problems. To compare the means of several groups under heteroskedasticity approximate tests are used in the literature. Weerahandi (1995a) introduced a test using the notion of generalized p-values for comparing the means of several populations when the variances are not equal. This test is referred to as a generalized F-test.

In this paper we compare the size performance of the Generalized F-test and four other widely used procedures: the Classical F-test for ANOVA, the F-test obtained by the weighted least-squares to adjust for heteroskedasticity, the Brown-Forsythe-test, and the Welch-test. The comparison is based on a simulation study of size performance of tests applied to the balanced one-way model. The intended level of the tests is set at 0.05. While the Generalized F-test was found to have size not exceeding the intended level, as heteroskedasticity becomes severe the other tests were found to have poor size performance. With mild heteroskedasticity the Welch-test and the classical ANOVA F-test have the intended levels, and the Welch-test was found to perform better than the latter. Widely used (due to computational convenience) weighted F-test was found to have very serious size problems. The size advantage of the generalized F-test was also found to be robust even under severe deviations from the assumption of normality.     

A K-SAMPLE EXTENSION TO FLIGNER'S CLASS OF TESTS FOR SCALE

The parameteric tests for equality of variance are well known. The classical F-test is typically used to test the hypothesis of equality of two variances, while tests such as those developed by Bartlett (1937) are commonly used for the k-sample hypothesis. These tests assume an underlying normal distribution and are quite sensitive to departures from normality (Box, 1953). Thus, when considering data that are from non-normal distributions, alternative nonparametric tests must be employed.
Fligner (1979) has proposed a class of two-sample distribution-free tests which possess very desirable properties and are attractive alternatives to other nonparametric tests for scale. The present paper extends the Fligner class of tests to the more general k-sample case.     

Nonparametric test for the homogeneity of the overall variability

In this paper, we propose a nonparametric test for homogeneity of overall variabilities for two multi-dimensional populations. Comparisons between the proposed nonparametric procedure and the asymptotic parametric procedure and a permutation test based on standardized generalized variances are made when the underlying populations are multivariate normal. We also study the performance of these test procedures when the underlying populations are non-normal. We observe that the nonparametric procedure and the permutation test based on standardized generalized variances are not as powerful as the asymptotic parametric test under normality. However, they are reliable and powerful tests for comparing overall variability under other multivariate distributions such as the multivariate Cauchy, the multivariate Pareto and the multivariate exponential distributions, even with small sample sizes. A Monte Carlo simulation study is used to evaluate the performance of the proposed procedures. An example from an educational study is used to illustrate the proposed nonparametric test.     

An empirical study of eight tests of partial correlation coefficients

Partial correlations can be used to statistically control for the effects of unwanted variables.Perhaps the most frequently used test of a partial correlation is the parametric F test,which requires normality of the joint distribution of observations.The possibility that this assumption may not be met in practice suggests a need for procedures that do not require normality.Unfortunately,the statistical literature provides little guidance for choosing other tests when the normalityassumption is not satisfied.Several nonparametric tests of partial correlations are investigated using a computer simulation study.Recommendations are made for selecting certain tests under particular conditions     

Comparison of Statistical Methods for Pretest–Posttest Designs in Terms of Type I Error Probability and Statistical Power

The pretest–posttest design is widely used to investigate the effect of an experimental treatment in biomedical research. The treatment effect may be assessed using analysis of variance (ANOVA) or analysis of covariance (ANCOVA). The normality assumption for parametric ANOVA and ANCOVA may be violated due to outliers and skewness of data. Nonparametric methods, robust statistics, and data transformation may be used to address the nonnormality issue. However, there is no simultaneous comparison for the four statistical approaches in terms of empirical type I error probability and statistical power. We studied 13 ANOVA and ANCOVA models based on parametric approach, rank and normal score-based nonparametric approach, Huber M-estimation, and Box–Cox transformation using normal data with and without outliers and lognormal data. We found that ANCOVA models preserve the nominal significance level better and are more powerful than their ANOVA counterparts when the dependent variable and covariate are correlated. Huber M-estimation is the most liberal method. Nonparametric ANCOVA, especially ANCOVA based on normal score transformation, preserves the nominal significance level, has good statistical power, and is robust for data distribution.     

Nearly exact tests in factorial experiments using the aligned rank transform

A procedure is studied that uses rank-transformed data to perform exact and estimated exact tests, which is an alternative to the commonly used F-ratio test procedure. First, a common parametric test statistic is computed using rank-transformed data, where two methods of ranking-ranks taken for the original observations and ranks taken after aligning the observations-are studied. Significance is then determined using either the exact permutation distribution of the statistic or an estimate of this distribution based on a random sample of all possible permutations. Simulation studies compare the performance of this method with the normal theory parametric F-test and the traditional rank transform procedure. Power and nominal type I error rates are compared under conditions when normal theory assumptions are satisfied, as well as when these assumptions are violated. The method is studied for a two-factor factorial arrangement of treatments in a completely randomized design and for a split-unit experiment. The power of the tests rivals the parametric F-test when normal theory assumptions are satisfied, and is usually superior when normal theory assumptions are not satisfied. Based on the evidence of this study, the exact aligned rank procedure appears to be the overall best choice for performing tests in a general factorial experiment.     

A closer examination on some parametric alternatives to the ANOVA F-test

In experiments, the classical (ANOVA) F-test is often used to test the omnibus null-hypothesis μ₁ = μ₂ ... = μ _j = ... = μ _n (all n population means are equal) in a one-way ANOVA design, even when one or more basic assumptions are being violated. In the first part of this article, we will briefly discuss the consequences of the different types of violations of the basic assumptions (dependent measurements, non-normality, heteroscedasticity) on the validity of the F-test. Secondly, we will present a simulation experiment, designed to compare the type I-error and power properties of both the F-test and some of its parametric adaptations: the Brown & Forsythe F^*-test and Welch’s V_w-test. It is concluded that the Welch V_w-test offers acceptable control over the type I-error rate in combination with (very) high power in most of the experimental conditions. Therefore, its use is highly recommended when one or more basic assumptions are being violated. In general, the use of the Brown & Forsythe F^*-test cannot be recommended on power considerations unless the design is balanced and the homoscedasticity assumption holds.     

Skewness by Splitting the Scale Parameter

This article describes a recursive nonparametric estimation for the local partial first derivative of an arbitrary function satisfied some regularity conditions and establishes its consistency and asymptotic normality under the assumption of strong mixing sequence. The proposed estimator is a variable window width version of the Watson-Nadaraya type of derivative estimator. The window width varied as more data points become available enables a recursive algorithm that reduce computational complexity from order N ³ normally required by batch methods for kernel regression to order N ². This approach is computationally simple and attractive from practical viewpoint especially when the situation call for frequent updating of first derivative estimates. For example, maintaining a delta-hedged position of a portfolio of equities with index options is one of many applications of such estimation.     

A two-sample empirical likelihood ratio test based on samples entropy

Powerful entropy-based tests for normality, uniformity and exponentiality have been well addressed in the statistical literature. The density-based empirical likelihood approach improves the performance of these tests for goodness-of-fit, forming them into approximate likelihood ratios. This method is extended to develop two-sample empirical likelihood approximations to optimal parametric likelihood ratios, resulting in an efficient test based on samples entropy. The proposed and examined distribution-free two-sample test is shown to be very competitive with well-known nonparametric tests. For example, the new test has high and stable power detecting a nonconstant shift in the two-sample problem, when Wilcoxon’s test may break down completely. This is partly due to the inherent structure developed within Neyman-Pearson type lemmas. The outputs of an extensive Monte Carlo analysis and real data example support our theoretical results. The Monte Carlo simulation study indicates that the proposed test compares favorably with the standard procedures, for a wide range of null and alternative distributions.     

The Multivariate Randomized Complete Block Design: A Novel Permutation Solution in Case of Ordered Categorical Variables

In this article, multivariate extensions of the combination-based test statistics for the comparison of several treatments in the multivariate Randomized Complete Block designs are introduced in case of categorical response variables. Several tests for the multivariate Randomized Complete Block designs, including MANOVA procedure, are compared with the method proposed via a Monte Carlo simulation study. The method has also been applied to a real case study in the field of sensorial testing studies. Results suggest that in each experimental situation where normality of the supposed underlying continuous model is hard to justify and especially when errors have heavy-tailed distributions, the proposed nonparametric procedure can be considered as a valid solution.     

Statistical inference for multivariate partially linear regression models

In this paper we study a class of multivariate partially linear regression models. Various estimators for the parametric component and the nonparametric component are constructed and their asymptotic normality established. In particular, we propose an estimator of the contemporaneous correlation among the multiple responses and develop a test for detecting the existence of such contemporaneous correlation without using any nonparametric estimation. The performance of the proposed estimators and test is evaluated through some simulation studies and an analysis of a real data set is used to illustrate the developed methodology. The Canadian Journal of Statistics 41: 1–22; 2013 © 2013 Statistical Society of Canada     

A control chart for variance based on squared ranks

A nonparametric control chart for variance is proposed. The chart is constructed following the change-point approach through the recursive use of the squared ranks test for variance. It is capable of detecting changes in the behaviour of individual observations with performance similar to a self-starting CUSUM chart for scale when normality is assumed, and a relatively better power when assessing nonnormal observations. A comparison is also made with two equivalent nonparametric charts based on Mood and Ansari-Bradley statistics. When dealing with symmetrical distributions, the proposed chart shows smaller (better) out-of-control average run length (ARL), and a competing performance otherwise. In addition, sensitivity to changes in mean and variance at the same time was tested. Extensive Monte Carlo simulation was used to measure performance, and a practical example is provided to illustrate how the proposed control chart can be implemented in practice.     

Nonparametric methods in factorial designs

Summary In this paper, we summarize some recent developments in the analysis of nonparametric models where the classical models of ANOVA are generalized in such a way that not only the assumption of normality is relaxed but also the structure of the designs is introduced in a broader framework and also the concept of treatment effects is redefined. The continuity of the distribution functions is not assumed so that not only data from continuous distributions but also data with ties are included in this general setup. In designs with independent observations as well as in repeated measures designs, the hypotheses are formulated by means of the distribution functions. The main results are given in a unified form. Some applications to special designs are considered, where in simple designs, some well known statistics (such as the Kruskal-Wallis statistic and the χ²-statistic for dichotomous data) come out as special cases. The general framework presented here enables the nonparametric analysis of data with continuous distribution functions as well as arbitrary discrete data such as count data, ordered categorical and dichotomous data. Received: October 13, 1999; revised version: June 26, 2000     

Simple and Exact Empirical Likelihood Ratio Tests for Normality Based on Moment Relations

The empirical likelihood (EL) technique is a powerful nonparametric method with wide theoretical and practical applications. In this article, we use the EL methodology in order to develop simple and efficient goodness-of-fit tests for normality based on the dependence between moments that characterizes normal distributions. The new empirical likelihood ratio (ELR) tests are exact and are shown to be very powerful decision rules based on small to moderate sample sizes. Asymptotic results related to the Type I error rates of the proposed tests are presented. We present a broad Monte Carlo comparison between different tests for normality, confirming the preference of the proposed method from a power perspective. A real data example is provided.     

A test of normality using nonparametrlic residuals

In this paper, we develop a test of the normality assumption of the errors using the residuals from a nonparametric kernel regression. Contrary to the existing tests based on the residuals from a parametric regression, our test is thus robust to misspecification of the regression function. The test statistic proposed here is a Bera-Jarque type test of skewness and kurtosis. We show that the test statistic has the usual x ²(2) limit distribution under the null hypothesis. In contrast to the results of Rilstone (1992), we provide a set of primitive assumptions that allow weakly dependent observations and data dependent bandwidth parameters. We also establish consistency property of the test. Monte Carlo experiments show that our test has reasonably good size and power performance in small samples and perfornu better than some of the alternative tests in various situations.     

A comparison of recent nonparametric methods for testing effects in two-by-two factorial designs

The two-way two-levels crossed factorial design is a commonly used design by practitioners at the exploratory phase of industrial experiments. The F-test in the usual linear model for analysis of variance (ANOVA) is a key instrument to assess the impact of each factor and of their interactions on the response variable. However, if assumptions such as normal distribution and homoscedasticity of errors are violated, the conventional wisdom is to resort to nonparametric tests. Nonparametric methods, rank-based as well as permutation, have been a subject of recent investigations to make them effective in testing the hypotheses of interest and to improve their performance in small sample situations. In this study, we assess the performances of some nonparametric methods and, more importantly, we compare their powers. Specifically, we examine three permutation methods (Constrained Synchronized Permutations, Unconstrained Synchronized Permutations and Wald-Type Permutation Test), a rank-based method (Aligned Rank Transform) and a parametric method (ANOVA-Type Test). In the simulations, we generate datasets with different configurations of distribution of errors, variance, factor's effect and number of replicates. The objective is to elicit practical advice and guides to practitioners regarding the sensitivity of the tests in the various configurations, the conditions under which some tests cannot be used, the tradeoff between power and type I error, and the bias of the power on one main factor analysis due to the presence of effect of the other factor. A dataset from an industrial engineering experiment for thermoformed packaging production is used to illustrate the application of the various methods of analysis, taking into account the power of the test suggested by the objective of the experiment.     

A CONSISTENT MODEL SPECIFICATION TEST BASED ON THE KERNEL SUM OF SQUARES OF RESIDUALS

This paper constructs a consistent model specification test based on the difference between the nonparametric kernel sum of squares of residuals and the sum of squares of residuals from a parametric null model. We establish the asymptotic normality of the proposed test statistic under the null hypothesis of correct parametric specification and show that the wild bootstrap method can be used to approximate the null distribution of the test statistic. Results from a small simulation study are reported to examine the finite sample performance of the proposed tests.     

A fiducial-based approach to the one-way ANOVA in the presence of nonnormality and heterogeneous error variances

In this study, we propose a new test for testing the equality of the treatment means in one-way ANOVA when the usual normality and the homogeneity of variances assumptions are not met. In developing the proposed test, we benefit from the Fisher's fiducial inference [1–3]. Distribution of the error terms is assumed to be long-tailed symmetric (LTS) which includes the normal distribution as a limiting case. Modified maximum likelihood (MML) estimators are used in the test statistics rather than the traditional least squares (LS) estimators, since LS estimators have very low efficiencies under nonnormal distributions, see Tiku [4] for the details of MML methodology. An extensive Monte Carlo simulation study is done to compare the efficiency of the proposed test with the corresponding test based on normal theory, see Li et al. [5]. Finally, we give a real life example to show the applicability of the proposed methodology.     

A density based empirical likelihood approach for testing bivariate normality

Sample entropy based tests, methods of sieves and Grenander estimation type procedures are known to be very efficient tools for assessing normality of underlying data distributions, in one-dimensional nonparametric settings. Recently, it has been shown that the density based empirical likelihood (EL) concept extends and standardizes these methods, presenting a powerful approach for approximating optimal parametric likelihood ratio test statistics, in a distribution-free manner. In this paper, we discuss difficulties related to constructing density based EL ratio techniques for testing bivariate normality and propose a solution regarding this problem. Toward this end, a novel bivariate sample entropy expression is derived and shown to satisfy the known concept related to bivariate histogram density estimations. Monte Carlo results show that the new density based EL ratio tests for bivariate normality behave very well for finite sample sizes. To exemplify the excellent applicability of the proposed approach, we demonstrate a real data example.     

The Effects of Misconceptions on the Properties of Friedman's Test

Friedman's test is a widely used rank-based alternative to the analysis of variance (ANOVA) F-test for identifying treatment differences in a randomized complete block design. Many texts provide incomplete or misleading information about when Friedman's test may be appropriately applied. We discuss the assumptions needed for the test and common misconceptions. We show via simulation that when the variance or skew of the treatment distributions differ, application of Friedman's test to detect differences in treatment location can result in Type I error probabilities larger than the nominal α, and even when α is unaffected, the power of the test can be less than expected.