Analysis of Type I Error Rates of Univariate and Multivariate Procedures in Repeated Measures Designs

We compared the robustness of univariate and multivariate statistical procedures to control Type I error rates when the normality and homocedasticity assumptions were not fulfilled. The procedures we evaluated are the mixed model adjusted by means of the SAS Proc Mixed module, and Bootstrap-F approach, Brown–Forsythe multivariate approach, Welch–James multivariate approach, and Welch–James multivariate approach with robust estimators. The results suggest that the Kenward Roger, Brown–Forsythe, Welch–James, and Improved Generalized Aprroximate procedures satisfactorily kept Type I error rates within the nominal levels for both the main and interaction effects under most of the conditions assessed.     

A closer examination on some parametric alternatives to the ANOVA F-test

In experiments, the classical (ANOVA) F-test is often used to test the omnibus null-hypothesis μ₁ = μ₂ ... = μ _j = ... = μ _n (all n population means are equal) in a one-way ANOVA design, even when one or more basic assumptions are being violated. In the first part of this article, we will briefly discuss the consequences of the different types of violations of the basic assumptions (dependent measurements, non-normality, heteroscedasticity) on the validity of the F-test. Secondly, we will present a simulation experiment, designed to compare the type I-error and power properties of both the F-test and some of its parametric adaptations: the Brown & Forsythe F^*-test and Welch’s V_w-test. It is concluded that the Welch V_w-test offers acceptable control over the type I-error rate in combination with (very) high power in most of the experimental conditions. Therefore, its use is highly recommended when one or more basic assumptions are being violated. In general, the use of the Brown & Forsythe F^*-test cannot be recommended on power considerations unless the design is balanced and the homoscedasticity assumption holds.     

A robust test for the homogeneity of scales

Many robust tests for the equality of variances have been proposed recently. Brown and Forsythe (1974) and Layard (1973) review some of the well-known procedures and compare them by simulation methods. Brown and Forsythe’s alternative formulation of Levene’s test statistic is found to be quite robust under certain nonnormal distributions. The performance of the methods, however, suffers in the presence of heavy tailed distributions such as the Cauchy distribution.

In this paper, we propose and study a simple robust test. The results obtained from the Monte Carlo study compare favorably with those of the existing procedures.     

Comparison of k independent,zero‐heavy lognormal distributions

Lachenbruch ( 1976 , 2001 ) introduced two‐part tests for comparison of two means in zero‐inflated continuous data. We are extending this approach and compare k independent distributions (by comparing their means, either overall or the departure from equal proportion of zeros and equal means of nonzero values) by introducing two tests: a two‐part Wald test and a two‐part likelihood ratio test. If the continuous part of the distributions is lognormal then the proposed two test statistics have asymptotically chi‐square distribution with $2(k-1)$ degrees of freedom. A simulation study was conducted to compare the performance of the proposed tests with several well‐known tests such as ANOVA, Welch ( 1951 ), Brown & Forsythe ( 1974 ), Kruskal–Wallis, and one‐part Wald test proposed by Tu & Zhou ( 1999 ). Results indicate that the proposed tests keep the nominal type I error and have consistently best power among all tests being compared. An application to rainfall data is provided as an example. The Canadian Journal of Statistics 39: 690–702; 2011. © 2011 Statistical Society of Canada     

A robust procedure for comparing several means under heteroscedasticity and nonnormality

By applying Tiku's MML robust procedure to Brown and Forsythe's (1974) statistic, this paper derives a robust and more powerful procedure for comparing several means under hetero-scedasticity and nonnormality. Some Monte Carlo studies indicate clearly that among five nonnormal distributions, except for the uniform distribution, the new test is more powerful than the Brown and Forsythe test under nonnormal distributions in all cases investigated and has substantially the same power as the Brown and Forsythe test under normal distribution.     

Robust tests for the equality of variances for clustered data

Tests for the equality of variances are often needed in applications. In genetic studies the assumption of equal variances of continuous traits, measured in identical and fraternal twins, is crucial for heritability analysis. To test the equality of variances of traits, which are non-normally distributed, Levene [H. Levene, Robust tests for equality of variances, in Contributions to Probability and Statistics, I. Olkin, ed. Stanford University Press, Palo Alto, California, 1960, pp. 278–292] suggested a method that was surprisingly robust under non-normality, and the procedure was further improved by Brown and Forsythe [M.B. Brown and A.B. Forsythe, Robust tests for the equality of variances, J. Amer. Statis. Assoc. 69 (1974), pp. 364–367]. These tests assumed independence of observations. However, twin data are clustered – observations within a twin pair may be dependent due to shared genes and environmental factors. Uncritical application of the tests of Brown and Forsythe to clustered data may result in much higher than nominal Type I error probabilities. To deal with clustering we developed an extended version of Levene's test, where the ANOVA step is replaced with a regression analysis followed by a Wald-type test based on a clustered version of the robust Huber–White sandwich estimator of the covariance matrix. We studied the properties of our procedure using simulated non-normal clustered data and obtained Type I error rates close to nominal as well as reasonable powers. We also applied our method to oral glucose tolerance test data obtained from a twin study of the metabolic syndrome and related components and compared the results with those produced by the traditional approaches.     

The 'improved' brown and forsythe test for mean equality: some things can't be fixed

Mehrotra (1997) presented an ‘;improved’ Brown and Forsythe (1974) statistic which is designed to provide a valid test of mean equality in independent groups designs when variances are heterogeneous. In particular, the usual Brown and Fosythe procedure was modified by using a Satterthwaite approximation for numerator degrees of freedom instead of the usual value of number of groups minus one. Mehrotra then, through Monte Carlo methods, demonstrated that the ‘improved’ method resulted in a robust test of significance in cases where the usual Brown and Forsythe method did not. Accordingly, this ‘improved’ procedure was recommended. We show that under conditions likely to be encountered in applied settings, that is, conditions involving heterogeneous variances as well as nonnormal data, the ‘improved’ Brown and Forsythe procedure results in depressed or inflated rates of Type I error in unbalanced designs. Previous findings indicate, however, that one can obtain a robust test by adopting a heteroscedastic statistic with the robust estimators, rather than the usual least squares estimators, and further improvement can be expected when critical significance values are obtained through bootstrapping methods.     

Testing Variability in the Two-Sample Case

A number of robust methods for testing variability have been reported in previous literature. An examination of these procedures for a wide variety of populations confirms their general robustness. Shoemaker's improvement of the F test extends that test use to a realistic variety of population shapes. However, a combination of the Brown–Forsythe and O'Brien methods based on testing kurtosis is shown to be conservative for a wide range of sample sizes and population distributions. The composite test is also shown to be more powerful in most conditions than other conservative procedures.     

Some comparative studies on testing parallelism of several straight lines under heteroscedastic variances

This paper examines the robustness of the Welch test, the James test as well as Tan's ANOVA test (to be referred as F_β test) for testing parallelism in k straight lines under heteroscedasticity and nonnormality. Results of Monte Carlo studies demonstrate the robustness of all tests with respect to departure from normality. Further, there is hardly any difference between these methods with respect to both power and size of the test.     

Testing methods for the one-way fixed effects ANOVA models of log-normal samples

For one-way fixed effects of log-normal data with unequal variance, the present study proposes a method to deal with heterogeneity. An appropriate hypothesis testing is demonstrated; and one of the approximate tests, such as the Alexander-Govern test, Welch test or James second-order test, is applied to control Type I error rate. Monte Carlo simulation is used to investigate the performance of the F test for log-scale, the F test for original scale, the James second-order test, the Welch test, and the Alexander-Govern test. The simulated results and real data analysis show that the proposed method is valid and powerful.     

Bayesian Alternatives to the F Test for Two Population Variances

Bayesian alternatives to the classical F test comparing two population variances are explored. Shoemaker (2003 Shoemaker , L. H. ( 2003 ). Fixing the F test for equal variances . The Amer. Statistician 57 : 105 – 114 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) suggested two adjustments to the F test due to it being very sensitive to the normal assumption of the two populations. A simulation study is performed to compare the Bayesian alternatives to the F test and Shoemaker's adjusted F tests as well as to the Levene/Brown–Forsythe and the squared rank nonparametric tests. The Bayesian alternatives assume a normal parent distribution and non informative priors and the conjugate prior for the variances; in addition, an exponential power distribution is considered as the parent distribution with a non informative prior for the variances. The latter looks to be very promising provided that a suitable value of a parameter which measures the extent of non normality is chosen.     

Serial correlation and distributions on the shere

Estimation and tests for serial correlation in recation and regression models with normal error have been derive from various points of view; for example: Anderson (1948), Durbi for Watson (1950, 1951, 1971), Theil (1965), Durbin (1970), Haq (1970), Kadiyala (1970), Abrahamse & Louter (1971), Levenbac (1972), Berenblut & Webb (1973), Phillips & Harvey (1974), a Sims (1975). In this paper we derive likelihood functions and most powerful tests for serial correclation in Locationa and regression models with arbitrary but specificed error; the methods extend to include the determination of the likelihood for the parameter of the error distribution.

In Section 2, we survey the modthods that have been used in deriving the various tests and estimates in the literature. In Section 2, we introduce the stataistical model that directly describes the error distribution and we obtain the likelihood function for error correlation and determine locally and specifically kost powerful tests for correlation. In Section 3 we consider the case with normal error derive a normal distribution on the sphere by radial projection. The likelihood function and test are then specialized to the case of normal error in Section 4. The computational procedures for the tests and related power functions are examined in Section 5. Power comparisons for the textile data of Theil and Nagar (1961), the consumption data of Kelin (1950), and the plums and the wheat data of Hildreth & Lu (1960) are presented in Section 6, while the likelihood functions for correlation in these data are given in Section 7.     

Adaptive bootstrap tests and its competitors in the c-sample scale problem

This paper deals with a study of different types of tests for the two-sided c-sample scale problem. We consider the classical parametric test of Bartlett [M.S. Bartlett, Properties of sufficiency and statistical tests, Proc. R. Stat. Soc. Ser. A. 160 (1937), pp. 268–282] several nonparametric tests, especially the test of Fligner and Killeen [M.A. Fligner and T.J. Killeen, Distribution-free two-sample tests for scale, J. Amer. Statist. Assoc. 71 (1976), pp. 210–213], the test of Levene [H. Levene, Robust tests for equality of variances, in Contribution to Probability and Statistics, I. Olkin, ed., Stanford University Press, Palo Alto, 1960, pp. 278–292] and a robust version of it introduced by Brown and Forsythe [M.B. Brown and A.B. Forsythe, Robust tests for the equality of variances, J. Amer. Statist. Assoc. 69 (1974), pp. 364–367] as well as two adaptive tests proposed by Büning [H. Büning, Adaptive tests for the c-sample location problem – the case of two-sided alternatives, Comm. Statist.Theory Methods. 25 (1996), pp. 1569–1582] and Büning [H. Büning, An adaptive test for the two sample scale problem, Nr. 2003/10, Diskussionsbeiträge des Fachbereich Wirtschaftswissenschaft der Freien Universität Berlin, Volkswirtschaftliche Reihe, 2003]. which are based on the principle of Hogg [R.V. Hogg, Adaptive robust procedures. A partial review and some suggestions for future applications and theory, J. Amer. Statist. Assoc. 69 (1974), pp. 909–927]. For all the tests we use Bootstrap sampling strategies, too. We compare via Monte Carlo Methods all the tests by investigating level α and power β of the tests for distributions with different strength of tailweight and skewness and for various sample sizes. It turns out that the test of Fligner and Killeen in combination with the bootstrap is the best one among all tests considered.     

Levene type tests for the ratio of two scales

Tests for the equality of variances are of interest in many areas such as quality control, agricultural production systems, experimental education, pharmacology, biology, as well as a preliminary to the analysis of variance, dose–response modelling or discriminant analysis. The literature is vast. Traditional non-parametric tests are due to Mood, Miller and Ansari–Bradley. A test which usually stands out in terms of power and robustness against non-normality is the W50 Brown and Forsythe [Robust tests for the equality of variances, J. Am. Stat. Assoc. 69 (1974), pp. 364–367] modification of the Levene test [Robust tests for equality of variances, in Contributions to Probability and Statistics, I. Olkin, ed., Stanford University Press, Stanford, 1960, pp. 278–292]. This paper deals with the two-sample scale problem and in particular with Levene type tests. We consider 10 Levene type tests: the W50, the M50 and L50 tests [G. Pan, On a Levene type test for equality of two variances, J. Stat. Comput. Simul. 63 (1999), pp. 59–71], the R-test [R.G. O'Brien, A general ANOVA method for robust tests of additive models for variances, J. Am. Stat. Assoc. 74 (1979), pp. 877–880], as well as the bootstrap and permutation versions of the W50, L50 and R tests. We consider also the F-test, the modified Fligner and Killeen [Distribution-free two-sample tests for scale, J. Am. Stat. Assoc. 71 (1976), pp. 210–213] test, an adaptive test due to Hall and Padmanabhan [Adaptive inference for the two-sample scale problem, Technometrics 23 (1997), pp. 351–361] and the two tests due to Shoemaker [Tests for differences in dispersion based on quantiles, Am. Stat. 49(2) (1995), pp. 179–182; Interquantile tests for dispersion in skewed distributions, Commun. Stat. Simul. Comput. 28 (1999), pp. 189–205]. The aim is to identify the effective methods for detecting scale differences. Our study is different with respect to the other ones since it is focused on resampling versions of the Levene type tests, and many tests considered here have not ever been proposed and/or compared. The computationally simplest test found robust is W50. Higher power, while preserving robustness, is achieved by considering the resampling version of Levene type tests like the permutation R-test (recommended for normal- and light-tailed distributions) and the bootstrap L50 test (recommended for heavy-tailed and skewed distributions). Among non-Levene type tests, the best one is the adaptive test due to Hall and Padmanabhan.     

Asyhptotic efficiency of sequential and non-sequential procedures based on fisher's method of combining tests

In this paper we propose test statistics based on Fisher's method of combining tests for hypotheses involving two or more parameters simultaneously, It Is shown that these tests are asymptotically efficient In the sense of Bahadur, It is then shown how these tests can be modified to give sequential test procedures which are efficient in the sense of Berk and Brown (1978).

The results in section 3 generalize the work of Perng (1977) and Durairajan (1980).     

A combined test for differences in scale based on the interquantile range

A class of tests due to Shoemaker (Commun Stat Simul Comput 28: 189–205, 1999) for differences in scale which is valid for a variety of both skewed and symmetric distributions when location is known or unknown is considered. The class is based on the interquantile range and requires that the population variances are finite. In this paper, we firstly propose a permutation version of it that does not require the condition of finite variances and is remarkably more powerful than the original one. Secondly we solve the question of what quantile choose by proposing a combined interquantile test based on our permutation version of Shoemaker tests. Shoemaker showed that the more extreme interquantile range tests are more powerful than the less extreme ones, unless the underlying distributions are very highly skewed. Since in practice you may not know if the underlying distributions are very highly skewed or not, the question arises. The combined interquantile test solves this question, is robust and more powerful than the stand alone tests. Thirdly we conducted a much more detailed simulation study than that of Shoemaker (1999) that compared his tests to the F and the squared rank tests showing that his tests are better. Since the F and the squared rank test are not good for differences in scale, his results suffer of such a drawback, and for this reason instead of considering the squared rank test we consider, following the suggestions of several authors, tests due to Brown–Forsythe (J Am Stat Assoc 69:364–367, 1974), Pan (J Stat Comput Simul 63:59–71, 1999), O’Brien (J Am Stat Assoc 74:877–880, 1979) and Conover et al. (Technometrics 23:351–361, 1981).     

The approximation of an unbiased test with a small level for the bioequivalence problem

In the bioequivalence problem. Brown. Hwang and Munk (1997) constructed an unbiased level a test and other tests which are uniformly more powerful than the two one-sided tests procedures when a iscomparatively larger. In this paper, for a small level, an unbiased test is shown to be approxirnately constructeQ lor tnis prooiem oy using tneir Metnog. ine numerical construction is also given.     

New monte carlo results on the robustness of the anova f,w and f statistics

Because the usual F test for equal means is not robust to unequal variances, Brown and Forsythe (1974a) suggest replacing F with the statistics F or W which are based on the Satterthwaite and Welch adjusted degrees of freedom procedures. This paper reports practical situations where both F and W give * unsatisfactory results. In particular, both F and W may not provide adequate control over Type I errors. Moreover, for equal variances, but unequal sample sizes, W should be avoided in favor of F (or F ), but for equal sample sizes, and possibly unequal variances, W was the only satisfactory statistic. New results on power are included as well. The paper also considers the effect of using F or W only after a significant test for equal variances has been obtained, and new results on the robustness of the F test are described. It is found that even for equal sample sizes as large as 50 per treatment group, there are practical situations where the F test does not provide adequately control over the probability of a Type I error.     

Robust analysis of variance: a simulation study

Traditional analysis-of-variance (ANOVA) is based on 'normality' and 'homogeneity' assumptions. If either or both of these assumptions are violated, then the one-way ANOVA may not be as powerful as robust analysis-of-variance (RANOVA) alternatives. We report the results of a simulation study of alternatives to ANOVA: Welch (W*), the first and second methods of James (J1*, 3J11*), Brown- Forsythe (BF*), a Box (B*) procedure, and the Kruskal-Wallis (KW*) procedure. Random samples from 14 distributions—uniform (0, 1), normal (0, 1), contaminated normal, SLATE, SLACU, SLASH, double exponential, Cauchy, half-normal, chi-squared (two degrees of freedom), chi-squared (four degrees of freedom) log normal, gamma (1, 2) and beta (2, 5)—were generated using a composite linear congruential generator. Corresponding test satis tics were computed and the empirical size for each test is given for three nominal a values (0.10, 0.05, 0.01). For k, we choose 3, 4 and 6. The sample sizes and combinations of sample sizes were chosen at 4, 6, 8, 10, 15 and 20. We then propose an adaptive algorithm based on an ancillary statistic that selects an ANOVA/RANOVA procedure for either symmetric or asymmetric data distributions, and for equal or unequal sample sizes.     

A non-parametric maximum test for the Behrens–Fisher problem

Non-normality and heteroscedasticity are common in applications. For the comparison of two samples in the non-parametric Behrens–Fisher problem, different tests have been proposed, but no single test can be recommended for all situations. Here, we propose combining two tests, the Welch t test based on ranks and the Brunner–Munzel test, within a maximum test. Simulation studies indicate that this maximum test, performed as a permutation test, controls the type I error rate and stabilizes the power. That is, it has good power characteristics for a variety of distributions, and also for unbalanced sample sizes. Compared to the single tests, the maximum test shows acceptable type I error control.