New monte carlo results on the robustness of the anova f,w and f statistics

Because the usual F test for equal means is not robust to unequal variances, Brown and Forsythe (1974a) suggest replacing F with the statistics F or W which are based on the Satterthwaite and Welch adjusted degrees of freedom procedures. This paper reports practical situations where both F and W give * unsatisfactory results. In particular, both F and W may not provide adequate control over Type I errors. Moreover, for equal variances, but unequal sample sizes, W should be avoided in favor of F (or F ), but for equal sample sizes, and possibly unequal variances, W was the only satisfactory statistic. New results on power are included as well. The paper also considers the effect of using F or W only after a significant test for equal variances has been obtained, and new results on the robustness of the F test are described. It is found that even for equal sample sizes as large as 50 per treatment group, there are practical situations where the F test does not provide adequately control over the probability of a Type I error.     

Two separate effects of variance heterogeneity on the validity and power of significance tests of location

Heterogeneity of variances of treatment groups influences the validity and power of significance tests of location in two distinct ways. First, if sample sizes are unequal, the Type I error rate and power are depressed if a larger variance is associated with a larger sample size, and elevated if a larger variance is associated with a smaller sample size. This well-established effect, which occurs in t and F tests, and to a lesser degree in nonparametric rank tests, results from unequal contributions of pooled estimates of error variance in the computation of test statistics. It is observed in samples from normal distributions, as well as non-normal distributions of various shapes. Second, transformation of scores from skewed distributions with unequal variances to ranks produces differences in the means of the ranks assigned to the respective groups, even if the means of the initial groups are equal, and a subsequent inflation of Type I error rates and power. This effect occurs for all sample sizes, equal and unequal. For the t test, the discrepancy diminishes, and for the Wilcoxon–Mann–Whitney test, it becomes larger, as sample size increases. The Welch separate-variance t test overcomes the first effect but not the second. Because of interaction of these separate effects, the validity and power of both parametric and nonparametric tests performed on samples of any size from unknown distributions with possibly unequal variances can be distorted in unpredictable ways.     

On the Weak Consistency of Permutation Tests

Consistency of some nonparametric tests with real variables has been studied by several authors under the assumption that population variance is finite and/or in the presence of some violations of the data exchangeability between samples. Since main inferential conclusions of permutation tests concern the actual dataset, where sample sizes are held fixed, we consider the notion of consistency in the weak version (in probability). Here, we characterize weak consistency of permutation tests assuming population mean is finite and without assuming existence of population variance. Moreover, since permutation test statistics do not require to be standardized, we do not assume that data are homoscedastic in the alternative. Several application examples to mostly used test statistics are discussed. A simulation study and some hints for robust testing procedures are also presented.     

A Weighted Least Squares Approach to Levene's Test of Homogeneity of Variance

In 1960 Levene suggested a potentially robust test of homogeneity of variance based on an ordinary least squares analysis of variance of the absolute values of mean-based residuals. Levene's test has since been shown to have inflated levels of significance when based on the F-distribution, and tests a hypothesis other than homogeneity of variance when treatments are unequally replicated, but the incorrect formulation is now standard output in several statistical packages. This paper develops a weighted least squares analysis of variance of the absolute values of both mean-based and median-based residuals. It shows how to adjust the residuals so that tests using the F -statistic focus on homogeneity of variance for both balanced and unbalanced designs. It shows how to modify the F -statistics currently produced by statistical packages so that the distribution of the resultant test statistic is closer to an F-distribution than is currently the case. The weighted least squares approach also produces component mean squares that are unbiased irrespective of which variable is used in Levene's test. To complete this aspect of the investigation the paper derives exact second-order moments of the component sums of squares used in the calculation of the mean-based test statistic. It shows that, for large samples, both ordinary and weighted least squares test statistics are equivalent; however they are over-dispersed compared to an F variable.     

Hypothesis Testing in Multivariate Linear Models with Randomly Missing Data

Tests for mean equality proposed by Weerahandi (1995) and Chen and Chen (1998), tests that do not require equality of population variances, were examined when data were not only heterogeneous but, as well, nonnormal in unbalanced completely randomized designs. Furthermore, these tests were compared to a test examined by Lix and Keselman (1998), a test that uses a heteroscedastic statistic (i.e., Welch, 1951) with robust estimators (20% trimmed means and Winsorized variances). Our findings confirmed previously published data that the tests are indeed robust to variance heterogeneity when the data are obtained from normal populations. However, the Weerahandi (1995) and Chen and Chen (1998) tests were not found to be robust when data were obtained from nonnormal populations. Indeed, rates of Type I error were typically in excess of 10% and, at times, exceeded 50%. On the other hand, the statistic presented by Lix and Keselman (1998) was generally robust to variance heterogeneity and nonnormality.     

Generalizations of barlett's and hartley's tests of homogeneity using “overall variability”

The generalized variance plays on important and useful role as a measure to compare overall variability of different populations in biological sciences (Goodman, 1968; Kocherlakota and Kocherlakota, 1983; Sokai, 1965). Here we present simple and elegant multivariate analogues to Bartlett's and Hartley's tests of homogeneity. Large sample distributions of the statistics are presented and the practical usefulness of the tests are demonstrated throught several examples.     

Hausman-type tests for individual and time effects in the panel regression model with incomplete data

By comparing estimators of the variance of idiosyncratic error at different robust levels, two Hausman-type test statistics are respectively constructed for the existence of individual and time effects in the panel regression model with incomplete data. The resultant test statistics have several desired properties. Firstly, they are robust to the presence of one effect when the other is tested. Secondly, they are immune to the non-normal distribution of the disturbances since the distributional conditions are not needed in the construction of the statistics. Thirdly, they have more robust performances than the main competitors in the literature when the covariates are correlated with the effects. Additionally, they are very simple and have no heavy computational burden. Joint tests for both of the two effects are also discussed. Monte Carlo evidence shows that the proposed tests have desired finite sample properties, and a real data analysis gives further support.     

On the role of minque in testing of hypotheses under hiked linear models

Testing of hypotheses under balanced ANOVA models is fairly simple and generally based on the usual ANOVA sums of squares. Difficulties may arise in special cases when these sums of squares do not form a complete sufficient statistic. There is a huge literature on this subject which was recently surveyed in Seifert's contribution to the book of Mumak (1904). But there are only a few results about unbalanced models. In such models the consideration of likelihood ratios leads to more complex sums of squares known from MINQUE theory.

Uniform optimality of testsusually reduces to local optimality. Here we prespnt a small review of methods proposed for testing of hypotheses in unbalanced models. where MINQUEI playb a major role. We discuss the use of iterated MINQUE for the construction of asymptotically optimal tests described in Humak (1984) and approximate tests based on locally uncorrelated linear combinations of MINQUE estimators by Seifert (1985), We show that the latter tests coincide with robust locally optimal invariant tests proposeci by Kariya and Sinha and Das and Sinha, if the number of variance components is two. Explicit expressions for corresponding tests are given for the unbalanced two-way cross classification random model, which covers some other models as special cases. A simulation study under lines the relevance of MINQUE for testing of hypotheses problems.     

Adjusted homogeneity tests of odds ratios when data are clustered

Three modified tests for homogeneity of the odds ratio for a series of 2 × 2 tables are studied when the data are clustered. In the case of clustered data, the standard tests for homogeneity of odds ratios ignore the variance inflation caused by positive correlation among responses of subjects within the same cluster, and therefore have inflated Type I error. The modified tests adjust for the variance inflation in the three existing standard tests: Breslow–Day, Tarone and the conditional score test. The degree of clustering effect is measured by the intracluster correlation coefficient, ρ. A variance correction factor derived from ρ is then applied to the variance estimator in the standard tests of homogeneity of the odds ratio. The proposed tests are an application of the variance adjustment method commonly used in correlated data analysis and are shown to maintain the nominal significance level in a simulation study. Copyright © 2004 John Wiley & Sons, Ltd.     

Variance Components Testing in ANOVA‐Type Mixed Models

The purpose of this article is threefold. First, variance components testing for ANOVA ‐type mixed models is considered, in which response may not be divided into independent sub‐vectors, whereas most of existing methods are for models where response can be divided into independent sub‐vectors. Second, testing that a certain subset of variance components is zero. Third, as normality is often violated in practice, it is desirable to construct tests under very mild assumptions. To achieve these goals, an adaptive difference‐based test and an adaptive trace‐based test are constructed. The test statistics are asymptotically normal under the null hypothesis, are consistent against all global alternatives and can detect local alternatives distinct from the null at a rate as close to n ^{? 1 ∕ 2} as possible with n being the sample size. Moreover, when the dimensions of variance components in different sets are bounded, we develop a test with chi‐square as its limiting null distribution. The finite sample performance of the tests is examined via simulations, and a real data set is analysed for illustration.     

Proportional subclass numbers in two-factor ANOVA

The between-within split of total sum of squares in one-way analysis of variance (ANOVA) is intuitively appealing and computationally simple, whether balanced or not. In the balanced two-factor setting, the same heuristic and computations apply to analyse treatment sum of squares into main effects and interaction effects sums of squares. Accomplishing the same in unbalanced settings is more difficult, requiring development of tests of general linear hypotheses. However, textbooks treat unbalanced settings with proportional subclasss numbers (psn) as essentially equivalent to balanced settings. It is shown here that, while psn permit an ANOVA-like partition of sums of squares, test statistics for main effects of the two factors generally test the wrong hypotheses when the model includes interaction effects.     

Testing methods for the one-way fixed effects ANOVA models of log-normal samples

For one-way fixed effects of log-normal data with unequal variance, the present study proposes a method to deal with heterogeneity. An appropriate hypothesis testing is demonstrated; and one of the approximate tests, such as the Alexander-Govern test, Welch test or James second-order test, is applied to control Type I error rate. Monte Carlo simulation is used to investigate the performance of the F test for log-scale, the F test for original scale, the James second-order test, the Welch test, and the Alexander-Govern test. The simulated results and real data analysis show that the proposed method is valid and powerful.     

A Distribution Free Test for the Equality of Scales

Situations where scale parameters are not nuisance factors to be controlled but outcomes to be explained arise in many contexts such as quality control, agricultural production systems, experimental education, the pharmaceutical industry and biology. Tests for homogeneity of variances are often of interest also as a preliminary to analysis of variance, dose-response modelling or discriminant analysis. The literature on tests for the equality of scales is vast. A test which usually stands out in terms of power and robustness against non normality is the modified Levene W50 test, however in the literature no test is found to be the most powerful one for every distribution. The goal of the article is to propose an effective method for comparing scales. More precisely, we propose a test for the equality of scales that, even though was not the most powerful one for every distribution, it has good overall performance under every type of distribution. This test has the form of a combined resampling test. It is important to note that non combined tests show good performance only in particular contexts. Size and power of the proposed test are studied via simulation and compared with many other robust tests for scale. A practical application to industrial quality control is discussed.     

A note on testing the nullity of the between group variance in the one-way random effects model under variance heterogeneity

In an unbalanced and heteroscedastic one-way random effects model, we compare, by way of simulation, several test statistics for testing the null hypothesis that the variance of the random effects, also named the between group variance, is zero. These tests are the classical F-test, the test proposed by Jeyaratnam & Othman, the Welch test, and a modified version of Welch's test.     

A note on homogeneity tests when combining 2×2 tables

This paper examines the use of homogeneity tests prior to tests of overall association among g 2 x 2 tables. When placed in the context of a one-way analysis of variance, hypotheses of overall association and homogeneity can be viewed as hypotheses regarding mean and treatment effects, respectively. In this context, the need for homogeneity tests is presented. What constitutes a relevant test of homogeneity is also examined. The conclusion is that some of the difficulties raised in the literature regarding tests of homogeneity stem from differences in the hypothesis of association being examined.     

Analysis of Variance by Randomization when Variances are Unequal

If there are significant factor and interaction effects with analysis of variance using ran-domization inference, they can be detected by tests that compare the F -statistics for the real data with the distributions of these statistics obtained by randomly allocating either the original observations or the residuals to the various factor combinations. Such tests involve the assumption that the effect of factors or interactions is to shift the observations for a factor combination by a fixed amount, without changing the amount of variation at that combination. In reality the expected amount of variation at each factor combination, as measured by the variance, may not be constant, which may upset the properties of the tests for the effects of factors and interactions. This paper discusses several possible methods for adjusting the randomization procedure to allow for this type of problem, including generalizations of methods that have been proposed for comparing the means of several samples when there is unequal variance but no factor structure. A simulation study shows that the best of the methods examined is one for which the randomized sets of data are designed to approximate the distributions of F -statistics when unequal variance is present.     

The Effect of a Variable Data Point on Hypothesis Tests for Means

The effect of a single variable data point, x, on the usual test statistics for traditional hypothesis tests for means is analyzed. It is shown that an outlier may have a profound and unexpected effect on the test statistic. Although it might appear that an outlier would tend to lend support to the alternate hypothesis, it may in fact detract from the significance of the test. In one-population tests and analysis of variance (ANOVA), the value of x that maximizes the significance of the test statistic is given. This value does not have to be unusually large or small. In fact, it often falls within the range of the other sample points. In the general one-population case, the limiting value for the test statistic is shown to be +1. In the case involving more than one population, it is shown that the limiting value of the test statistic is a function only of the number of members in the samples and not their relative values. Special cases are identified in which the test statistic is shown to have unique characteristics depending on the characteristics of the data.     

Comparison of k independent,zero‐heavy lognormal distributions

Lachenbruch ( 1976 , 2001 ) introduced two‐part tests for comparison of two means in zero‐inflated continuous data. We are extending this approach and compare k independent distributions (by comparing their means, either overall or the departure from equal proportion of zeros and equal means of nonzero values) by introducing two tests: a two‐part Wald test and a two‐part likelihood ratio test. If the continuous part of the distributions is lognormal then the proposed two test statistics have asymptotically chi‐square distribution with $2(k-1)$ degrees of freedom. A simulation study was conducted to compare the performance of the proposed tests with several well‐known tests such as ANOVA, Welch ( 1951 ), Brown & Forsythe ( 1974 ), Kruskal–Wallis, and one‐part Wald test proposed by Tu & Zhou ( 1999 ). Results indicate that the proposed tests keep the nominal type I error and have consistently best power among all tests being compared. An application to rainfall data is provided as an example. The Canadian Journal of Statistics 39: 690–702; 2011. © 2011 Statistical Society of Canada     

Robust analysis of variance

SUMMARY For the c -sample location problem with equal and unequal variances, we compare the classical F -test and its robustified version-the Welch test-with some nonparametric counterparts defined for two-sided and one-sided ordered alternatives, such as trend and umbrella alternatives. A new rank test for long-tailed distributions is proposed. The comparison is referred to level alpha and power beta of the tests, and is carried out via Monte Carlo simulation, assuming short-, medium- and long-tailed as well as asymmetric distributions. It turns out that the Welch test is the best one in the case of unequal variances but in the case of equal variances special non-parametric tests are to prefer.     

A comparison of test statistics for the recovery of rapid growth‐based enumeration tests

This paper considers five test statistics for comparing the recovery of a rapid growth‐based enumeration test with respect to the compendial microbiological method using a specific nonserial dilution experiment. The finite sample distributions of these test statistics are unknown, because they are functions of correlated count data. A simulation study is conducted to investigate the type I and type II error rates. For a balanced experimental design, the likelihood ratio test and the main effects analysis of variance (ANOVA) test for microbiological methods demonstrated nominal values for the type I error rate and provided the highest power compared with a test on weighted averages and two other ANOVA tests. The likelihood ratio test is preferred because it can also be used for unbalanced designs. It is demonstrated that an increase in power can only be achieved by an increase in the spiked number of organisms used in the experiment. The power is surprisingly not affected by the number of dilutions or the number of test samples. A real case study is provided to illustrate the theory. Copyright © 2013 John Wiley & Sons, Ltd.