A robust test for the homogeneity of scales

Many robust tests for the equality of variances have been proposed recently. Brown and Forsythe (1974) and Layard (1973) review some of the well-known procedures and compare them by simulation methods. Brown and Forsythe’s alternative formulation of Levene’s test statistic is found to be quite robust under certain nonnormal distributions. The performance of the methods, however, suffers in the presence of heavy tailed distributions such as the Cauchy distribution.

In this paper, we propose and study a simple robust test. The results obtained from the Monte Carlo study compare favorably with those of the existing procedures.     

The 'improved' brown and forsythe test for mean equality: some things can't be fixed

Mehrotra (1997) presented an ‘;improved’ Brown and Forsythe (1974) statistic which is designed to provide a valid test of mean equality in independent groups designs when variances are heterogeneous. In particular, the usual Brown and Fosythe procedure was modified by using a Satterthwaite approximation for numerator degrees of freedom instead of the usual value of number of groups minus one. Mehrotra then, through Monte Carlo methods, demonstrated that the ‘improved’ method resulted in a robust test of significance in cases where the usual Brown and Forsythe method did not. Accordingly, this ‘improved’ procedure was recommended. We show that under conditions likely to be encountered in applied settings, that is, conditions involving heterogeneous variances as well as nonnormal data, the ‘improved’ Brown and Forsythe procedure results in depressed or inflated rates of Type I error in unbalanced designs. Previous findings indicate, however, that one can obtain a robust test by adopting a heteroscedastic statistic with the robust estimators, rather than the usual least squares estimators, and further improvement can be expected when critical significance values are obtained through bootstrapping methods.     

Tests for homogeneity of variance

Six procedures which convert tests of homogeneity of variance into tests for mean equality for independent groups are compared. The tests are the analysis of variance (ANOVA) and Welch F statistics. The Welch statistics are included since it was anticipated that ANOVA would not provide a robust test when samples of unequal sizes are obtained from non-normal populations. However, the Welch tests are not found to be uniformly preferrable. In addition, a prior recommendation for Miller's jackknife procedure is not supported for the unequal sample size case. The data indicates that the current tests for variance heterogeneity are either sensitive to non-normality or, if robust, lacking in power. Therefore, these tests cannot be recommended for the purpose of testing the validity of the ANOVA homogeneity assumption.     

Size performance of some tests in one-way anova

Tsui and Weerahandi (1989) introduced the notion of generalized p-values and since then this idea is used to solve many statistical testing problems. Heteroskedasticity is one of the major practical problems encountered in ANOVA problems. To compare the means of several groups under heteroskedasticity approximate tests are used in the literature. Weerahandi (1995a) introduced a test using the notion of generalized p-values for comparing the means of several populations when the variances are not equal. This test is referred to as a generalized F-test.

In this paper we compare the size performance of the Generalized F-test and four other widely used procedures: the Classical F-test for ANOVA, the F-test obtained by the weighted least-squares to adjust for heteroskedasticity, the Brown-Forsythe-test, and the Welch-test. The comparison is based on a simulation study of size performance of tests applied to the balanced one-way model. The intended level of the tests is set at 0.05. While the Generalized F-test was found to have size not exceeding the intended level, as heteroskedasticity becomes severe the other tests were found to have poor size performance. With mild heteroskedasticity the Welch-test and the classical ANOVA F-test have the intended levels, and the Welch-test was found to perform better than the latter. Widely used (due to computational convenience) weighted F-test was found to have very serious size problems. The size advantage of the generalized F-test was also found to be robust even under severe deviations from the assumption of normality.     

Robust statistics for testing equality of means or variances

For testing the equality of means (location parameters) of two populations, Tiku (1980a) defined a statistic Tc (based on symmetrically censored samples) and showed that this statistic is robust to underlying populations and is also remarkably powerful. In this paper, we define a similar statistic T (based on samples s with observations censored only on one side) and show that this stat is tic is more powerful than T and nonparametric statistics, C for skew populations. We also provide a modification of this statistic for testing the equality of two population variances.     

On testing homogeneity of variances for nonnormal models using entropy

This paper deals with testing equality of variances of observations in the different treatment groups assuming treatment effects are fixed. We study the distribution of a test statistic which is known to perform comparably well with other statistics for the same purpose under normality. The statistic we consider is based on Shannon’s entropy for a distribution function. We will derive the asymptotic expansion for the distribution of the test statistic based on Shannon’s entropy under nonnormality and numerically examine its performance in comparison with the modified likelihood ratio criteria for normal and some nonnormal populations.      

Contrast analysis for scale differences

Research on tests for scale equality, that are robust to violations of the distributional normality assumption, have focused exclusively on an overall test statistic and have not examined procedures for identifying specific differences in multiple group designs. The present study compares four contrast analysis procedures for scale differences in the single factor four group design. Two data transformations are considered under several conbinations of variance difference, sample sizes, and distributional forms.The results indicate that no single transformation or analysis procedure is uniformly superior in controlling the familywise error rate or in statistical power. The relationship between sample size and variances is a major factor in selecting a contrast analysis procedure.     

Levels of significance of some two-sample tests wkem observations are from compound normal distributions

This paper presents an investigation of the behavior of the levels of significance of the two-sample t and its related tests and the Mann-Whitney test when the samples are randomly drawn from mixtures of two normal populations (compound normals) and when the sample sizes are small (combined sample sizes ? 15). The use of the compound normal allows for investigation when the underlying populations are unequal, nonnormal, heterogeneous in variances, unimodal or bimodal, possessing smaller than normal kurtosis or containing contamination. The exact distribution of the t and its related tests are given. However, they are not readily amenable to calculations. Most of the numerical results presented were obtained by simulations     

Robust tests for the common principal components model

When dealing with several populations, the common principal components (CPC) model assumes equal principal axes but different variances along them. In this paper, a robust log-likelihood ratio statistic allowing to test the null hypothesis of a CPC model versus no restrictions on the scatter matrices is introduced. The proposal plugs into the classical log-likelihood ratio statistic robust scatter estimators. Using the same idea, a robust log-likelihood ratio and a robust Wald-type statistic for testing proportionality against a CPC model are considered. Their asymptotic distributions under the null hypothesis and their partial influence functions are derived. A small simulation study allows to compare the behavior of the classical and robust tests, under normal and contaminated data.     

A stein two-sample procedure for the general linear model with unequal error variances

The pronerties of the tests and confidence regions for the parameters in the classical general linear model depend upon the equality of the variances of the error terms. The level and power of tests and the confidence coefficients associated with confidence regions are vitiated when the assumption of equality is not true. Even when the error variances are equal the power of tests and the size of confidence regions depend upon the unknown common variance and hence are uncontrollable. This paper presents a two-stage procedure which yields tests and confidence regions which are completely independent of the variances of the errors and hence tests with controllable power and confidence regions of fixed controllable size are obtained.     

An update on ‘a comparative study of tests for homogeneity of variance’

Tests for equality of variances using independent samples are widely used in data analysis. Conover et al. [A comparative study of tests for homogeneity of variance, with applications to the outer continental shelf bidding data. Technometrics. 1981;23:351–361], won the Youden Prize by comparing 56 variations of popular tests for variance on the basis of robustness and power in 60 different scenarios. None of the tests they compared were robust and powerful for the skewed distributions they considered. This study looks at 12 variations they did not consider, and shows that 10 are robust for the skewed distributions they considered plus the lognormal distribution, which they did not study. Three of these 12 have clearly superior power for skewed distributions, and are competitive in terms of robustness and power for all of the distributions considered. They are recommended for general use based on robustness, power, and ease of application.     

Robust test for means when population variances are unequal

We consider the problem of testing the equality of two population means when the population variances are not necessarily equal. We propose a Welch-type statistic, say T^* _c, based on Tiku!s ‘1967, 1980’ modified maximum likelihood estimators, and show that this statistic is robust to symmetric and moderately skew distributions. We investigate the power properties of the statistic T^* _c; T^* _c clearly seems to be more powerful than Yuen's ‘1974’ Welch-type robust statistic based on the trimmed sample means and the matching sample variances. We show that the analogous statistics based on the ‘adaptive’ robust estimators give misleading Type I errors. We generalize the results to testing linear contrasts among k population means     

Multi-sample test for high-dimensional covariance matrices

In this paper, we propose a new test statistic for testing the equality of high-dimensional covariance matrices for multiple populations. The proposed test statistic generalizes the test of the equality of two population covariance matrices proposed by Li and Chen (2012).     

Comparison of nonparametric analysis of variance methods: A vote for van der Waerden

ABSTRACT

For two-way layouts in a between-subjects analysis of variance design, the parametric F-test is compared with seven nonparametric methods: rank transform (RT), inverse normal transform (INT), aligned rank transform (ART), a combination of ART and INT, Puri & Sen's L statistic, Van der Waerden, and Akritas and Brunners ANOVA-type statistics (ATS). The type I error rates and the power are computed for 16 normal and nonnormal distributions, with and without homogeneity of variances, for balanced and unbalanced designs as well as for several models including the null and the full model. The aim of this study is to identify a method that is applicable without too much testing for all the attributes of the plot. The Van der Waerden test shows the overall best performance though there are some situations in which it is disappointing. The Puri & Sen's and the ATS tests show generally very low power. These two and the other methods cannot keep the type I error rate under control in too many situations. Especially in the case of lognormal distributions, the use of any of the rank-based procedures can be dangerous for cell sizes above 10. As already shown by many other authors, nonnormal distributions do not violate the parametric F-test, but unequal variances do, and heterogeneity of variances leads to an inflated error rate more or less also for the nonparametric methods. Finally, it should be noted that some procedures show rising error rates with increasing cell sizes, the ART, especially for discrete variables, and the RT, Puri & Sen, and the ATS in the cases of heteroscedasticity.     

Diagnosing explainable heterogeneity of variance in random‐effects models

Data‐analytic tools for models other than the normal linear regression model are relatively rare. Here we develop plots and diagnostic statistics for nonconstant variance for the random‐effects model (REM). REMs for longitudinal data include both within‐ and between‐subject variances. A basic assumption is that the two variance terms are constant across subjects. However, we often find that these variances are functions of covariates, and the data set has what we call explainable heterogeneity, which needs to be allowed for in the model. We characterize several types of heterogeneity of variance in REMs and develop three diagnostic tests using the score statistic: one for each of the two variance terms, and the third for a form of multivariate nonconstant variance. For each test we present an adjusted residual plot which can identify cases that are unusually influential on the outcome of the test.     

A K-SAMPLE EXTENSION TO FLIGNER'S CLASS OF TESTS FOR SCALE

The parameteric tests for equality of variance are well known. The classical F-test is typically used to test the hypothesis of equality of two variances, while tests such as those developed by Bartlett (1937) are commonly used for the k-sample hypothesis. These tests assume an underlying normal distribution and are quite sensitive to departures from normality (Box, 1953). Thus, when considering data that are from non-normal distributions, alternative nonparametric tests must be employed.
Fligner (1979) has proposed a class of two-sample distribution-free tests which possess very desirable properties and are attractive alternatives to other nonparametric tests for scale. The present paper extends the Fligner class of tests to the more general k-sample case.     

Comparison of ANOVA-F and ANOM tests with regard to type I error rate and test power

A Monte Carlo simulation was conducted to compare the type I error rate and test power of the analysis of means (ANOM) test to the one-way analysis of variance F-test (ANOVA-F). Simulation results showed that as long as the homogeneity of the variance assumption was satisfied, regardless of the shape of the distribution, number of group and the combination of observations, both ANOVA-F and ANOM test have displayed similar type I error rates. However, both tests have been negatively affected from the heterogeneity of the variances. This case became more obvious when the variance ratios increased. The test power values of both tests changed with respect to the effect size (Δ), variance ratio and sample size combinations. As long as the variances are homogeneous, ANOVA-F and ANOM test have similar powers except unbalanced cases. Under unbalanced conditions, the ANOVA-F was observed to be powerful than the ANOM-test. On the other hand, an increase in total number of observations caused the power values of ANOVA-F and ANOM test approach to each other. The relations between effect size (Δ) and the variance ratios affected the test power, especially when the sample sizes are not equal. As ANOVA-F has become to be superior in some of the experimental conditions being considered, ANOM is superior in the others. However, generally, when the populations with large mean have larger variances as well, ANOM test has been seen to be superior. On the other hand, when the populations with large mean have small variances, generally, ANOVA-F has observed to be superior. The situation became clearer when the number of the groups is 4 or 5.     

A goodness-of-fit test for overdispersed binomial (or multinomial) models

Overdispersion or extra variation is a common phenomenon that occurs when binomial (multinomial) data exhibit larger variances than that permitted by the binomial (multinomial) model. This arises when the data are clustered or when the assumption of independence is violated. Goodness-of-fit (GOF) tests available in the overdispersion literature have focused on testing for the presence of overdispersion in the data and hence they are not applicable for choosing between the several competing overdispersion models. In this paper, we consider a GOF test proposed by Neerchal and Morel [1998. Large cluster results for two parametric multinomial extra variation models. J. Amer. Statist. Assoc. 93(443), 1078–1087], and study its distributional properties and performance characteristics. This statistic is a direct analogue of the usual Pearson chi-squared statistic, but is also applicable when the clusters are not necessarily of the same size. As this test statistic is for testing model adequacy against the alternative that the model is not adequate, it is applicable in testing two competing overdispersion models.     

A comparison of selection procedures for selecting the best normal population under heterogeneity of variance

This study examines the comparative probabilities of making a correct selection when using the means procedure (M), the medians procedure (D) and the rank-sum procedure (S) to correctly select the normal population with the largest mean under heterogeneity of variance. The comparison is conducted by using Monte-Carlo simulation techniques for 3, 4, and 5 normal populations under the condition that equal sample sizes are taken from each population. The population means and standard deviations are assumed to be equally-spaced. Two types of heterogeneity of variance are considered: (1) associating larger means with larger variances, and (2) associating larger means with smaller variances.     

Some solutions to the multivariate Behrens–Fisher problem for dissimilarity‐based analyses

The essence of the generalised multivariate Behrens–Fisher problem (BFP) is how to test the null hypothesis of equality of mean vectors for two or more populations when their dispersion matrices differ. Solutions to the BFP usually assume variables are multivariate normal and do not handle high‐dimensional data. In ecology, species' count data are often high‐dimensional, non‐normal and heterogeneous. Also, interest lies in analysing compositional dissimilarities among whole communities in non‐Euclidean (semi‐metric or non‐metric) multivariate space. Hence, dissimilarity‐based tests by permutation (e.g., PERMANOVA, ANOSIM) are used to detect differences among groups of multivariate samples. Such tests are not robust, however, to heterogeneity of dispersions in the space of the chosen dissimilarity measure, most conspicuously for unbalanced designs. Here, we propose a modification to the PERMANOVA test statistic, coupled with either permutation or bootstrap resampling methods, as a solution to the BFP for dissimilarity‐based tests. Empirical simulations demonstrate that the type I error remains close to nominal significance levels under classical scenarios known to cause problems for the un‐modified test. Furthermore, the permutation approach is found to be more powerful than the (more conservative) bootstrap for detecting changes in community structure for real ecological datasets. The utility of the approach is shown through analysis of 809 species of benthic soft‐sediment invertebrates from 101 sites in five areas spanning 1960 km along the Norwegian continental shelf, based on the Jaccard dissimilarity measure.