Two separate effects of variance heterogeneity on the validity and power of significance tests of location

Heterogeneity of variances of treatment groups influences the validity and power of significance tests of location in two distinct ways. First, if sample sizes are unequal, the Type I error rate and power are depressed if a larger variance is associated with a larger sample size, and elevated if a larger variance is associated with a smaller sample size. This well-established effect, which occurs in t and F tests, and to a lesser degree in nonparametric rank tests, results from unequal contributions of pooled estimates of error variance in the computation of test statistics. It is observed in samples from normal distributions, as well as non-normal distributions of various shapes. Second, transformation of scores from skewed distributions with unequal variances to ranks produces differences in the means of the ranks assigned to the respective groups, even if the means of the initial groups are equal, and a subsequent inflation of Type I error rates and power. This effect occurs for all sample sizes, equal and unequal. For the t test, the discrepancy diminishes, and for the Wilcoxon–Mann–Whitney test, it becomes larger, as sample size increases. The Welch separate-variance t test overcomes the first effect but not the second. Because of interaction of these separate effects, the validity and power of both parametric and nonparametric tests performed on samples of any size from unknown distributions with possibly unequal variances can be distorted in unpredictable ways.     

Statistical considerations in bioequivalence of two area under the concentration–time curves obtained from serial sampling data

In this paper, we study the bioequivalence (BE) inference problem motivated by pharmacokinetic data that were collected using the serial sampling technique. In serial sampling designs, subjects are independently assigned to one of the two drugs; each subject can be sampled only once, and data are collected at K distinct timepoints from multiple subjects. We consider design and hypothesis testing for the parameter of interest: the area under the concentration–time curve (AUC). Decision rules in demonstrating BE were established using an equivalence test for either the ratio or logarithmic difference of two AUCs. The proposed t-test can deal with cases where two AUCs have unequal variances. To control for the type I error rate, the involved degrees-of-freedom were adjusted using Satterthwaite's approximation. A power formula was derived to allow the determination of necessary sample sizes. Simulation results show that, when the two AUCs have unequal variances, the type I error rate is better controlled by the proposed method compared with a method that only handles equal variances. We also propose an unequal subject allocation method that improves the power relative to that of the equal and symmetric allocation. The methods are illustrated using practical examples.     

Testing for equivalence of variances using Hartley's ratio

Hartley's test for homogeneity of k normal‐distribution variances is based on the ratio between the maximum sample variance and the minimum sample variance. In this paper, the author uses the same statistic to test for equivalence of k variances. Equivalence is defined in terms of the ratio between the maximum and minimum population variances, and one concludes equivalence when Hartley's ratio is small. Exact critical values for this test are obtained by using an integral expression for the power function and some theoretical results about the power function. These exact critical values are available both when sample sizes are equal and when sample sizes are unequal. One related result in the paper is that Hartley's test for homogeneity of variances is no longer unbiased when the sample sizes are unequal. The Canadian Journal of Statistics 38: 647–664; 2010 © 2010 Statistical Society of Canada     

Parametric Bootstrap Tests for Unbalanced Three-factor Nested Designs under Heteroscedasticity

In this article, we consider the three-factor unbalanced nested design model without the assumption of equal error variance. For the problem of testing “main effects” of the three factors, we propose a parametric bootstrap (PB) approach and compare it with the existing generalized F (GF) test. The Type I error rates of the tests are evaluated using Monte Carlo simulation. Our studies show that the PB test performs better than the generalized F-test. The PB test performs very satisfactorily even for small samples while the GF test exhibits poor Type I error properties when the number of factorial combinations or treatments goes up. It is also noted that the same tests can be used to test the significance of the random effect variance component in a three-factor mixed effects nested model under unequal error variances.     

Simultaneous confidence intervals for pairwise multiple comparisons in a two-way unbalanced design with unequal variances

In this research, we propose simultaneous confidence intervals for all pairwise multiple comparisons in a two-way unbalanced design with unequal variances, using a parametric bootstrap approach. Simulation results show that Type 1 error of the multiple comparison test is close to the nominal level even for small samples. They also show that the proposed method outperforms Tukey–Kramer procedure when variances are heteroscedastic and group sizes are unequal.     

Heterogeneity of variance and biased hypothesis tests

This study examined the influence of heterogeneity of variance on Type I error rates and power of the independent-samples Student's t-test of equality of means on samples of scores from normal and 10 non-normal distributions. The same test of equality of means was performed on corresponding rank-transformed scores. For many non-normal distributions, both versions produced anomalous power functions, resulting partly from the fact that the hypothesis test was biased, so that under some conditions, the probability of rejecting H ₀ decreased as the difference between means increased. In all cases where bias occurred, the t-test on ranks exhibited substantially greater bias than the t-test on scores. This anomalous result was independent of the more familiar changes in Type I error rates and power attributable to unequal sample sizes combined with unequal variances.     

The power functions of the likelihood ratio tests for a simple tree ordering in normal means: unequal weights

Likelihood ratio tests are considered for two testing situations; testing for the homogeneity of k normal means against the alternative restricted by a simple tree ordering trend and testing the null hypothesis that the means satisfy the trend against all alternatives. Exact expressions are given for the power functions for k = 3 and 4 and unequal sample sizes, both for the case of known and unknown population variances, and approximations are discussed for larger k. Also, Bartholomew’s conjectures concerning minimal and maximal powers are investigated for the case of equal and unequal sample sizes. The power formulas are used to compute powers for a numerical example.     

A step-down heteroscedastic multiple comparison procedure

All-pairs power in a one-way ANOVA is the probability of detecting all true differences between pairs of means. Ramsey (1978) found that for normal distributions having equal variances, step-down multiple comparison procedures can have substantially more all-pairs power than single-step procedures, such as Tukey’s HSD, when equal sample sizes are randomly sampled from each group. This paper suggests a step-down procedure for the case of unequal variances and compares it to Dunnett's T3 technique. The new procedure is similar in spirit to one of the heteroscedastic procedures described by Hochberg and Tamhane (1987), but it has certain advantages that are discussed in the paper. Included are results on unequal sample sizes.     

Robustness of the two - sample t - test under violations of the homogeneity of variance assumption,part ii

In a previous paper, Posten, Yeh and Owen (1982), the robustness of the type I error for the two tailed two sample t - test was studied under departures from the assumption of equal variances. The level of robustness of this test was then quantified under the concept of regions of robustness. These results are extended here to the one - tailed test for the same problem. The high level of robustness for equal or nearly equal sample sizes observed in the previous study is again documented quantitatively.     

The two-sample t test versus satterthwaite's approximate f test

A comparison between the two-sample t test and Satterthwaite's approximate F test is made, assuming the choice between these two tests is based on a preliminary test on the variances. Exact formulas for the sizes and powers of the tests are derived. Sizes and powers are then calculated and compared for several situations.     

A Test for the Equality of Parameters for Separate Regression Models in the Presence of Heteroskedasticity

Testing for the equality of regression coefficients across two regressions is a problem considered by analysts in a variety of fields. If the variances of the errors of the two regressions are not equal, then it is known that the standard large sample F-test used to test the equality of the coefficients is compromised by the fact that its actual size can differ substantially from the stated level of significance in small samples. This article addresses this problem and borrows from the literature on the Behrens-Fisher problem to provide some simple modifications of the large sample test which allows one to better control the probability of committing a Type I error. Empirical evidence is presented which indicates that the suggested modifications provide tests which are superior to well-known alternative tests over a wide range of the parameter space.     

A test statistic based on ranked set sampling for two normal means

In this study, we considered a hypothesis test for the difference of two population means using ranked set sampling. We proposed a test statistic for this hypothesis test with more than one cycle under normality. We also investigate the performance of this test statistic, when the assumptions hold and are violated. For this reason, we investigate the type I error and power rates of tests under normality with equal and unequal variances, non-normality with equal and unequal variances. We also examine the performance of this test under imperfect ranking case. The simulation results show that derived test performs quite well.     

A Parametric Bootstrap Test for Two-Way ANOVA Model Without Interaction Under Heteroscedasticity

In this article we consider the two-way ANOVA model without interaction under heteroscedasticity. For the problem of testing equal effects of factors, we propose a parametric bootstrap (PB) approach and compare it with existing the generalized F (GF) test. The Type I error rates and powers of the tests are evaluated using Monte Carlo simulation. Our studies show that the PB test performs better than the GF test. The PB test performs very satisfactorily even for small samples while the GF test exhibits poor Type I error properties when the number of factorial combinations or treatments goes up. It is also noted that the same tests can be used to test the significance of random effect variance component in a two-way mixed-effects model under unequal error variances.     

Interval Estimation for the Difference of Two Independent Variances

Analytical methods for interval estimation of differences between variances have not been described. A simple analytical method is given for interval estimation of the difference between variances of two independent samples. It is shown, using simulations, that confidence intervals generated with this method have close to nominal coverage even when sample sizes are small and unequal and observations are highly skewed and leptokurtic, provided the difference in variances is not very large. The method is also adapted for testing the hypothesis of no difference between variances. The test is robust but slightly less powerful than Bonett's test with small samples.     

Many-to-one comparison of nonlinear growth curves for Washington's Red Delicious apple

In this article, we are interested in comparing growth curves for the Red Delicious apple in several locations to that of a reference site. Although such multiple comparisons are common for linear models, statistical techniques for nonlinear models are not prolific. We theoretically derive a test statistic, considering the issues of sample size and design points. Under equal sample sizes and same design points, our test statistic is based on the maximum of an equi-correlated multivariate chi-square distribution. Under unequal sample sizes and design points, we derive a general correlation structure, and then utilize the multivariate normal distribution to numerically compute critical points for the maximum of the multivariate chi-square. We apply this statistical technique to compare the growth of Red Delicious apples at six locations to a reference site in the state of Washington in 2009. Finally, we perform simulations to verify the performance of our proposed procedure for Type I error and marginal power. Our proposed method performs well in regard to both.     

Multiple comparisons based on a modified one-step M-estimator

Although many methods are available for performing multiple comparisons based on some measure of location, most can be unsatisfactory in at least some situations, in simulations when sample sizes are small, say less than or equal to twenty. That is, the actual Type I error probability can substantially exceed the nominal level, and for some methods the actual Type I error probability can be well below the nominal level, suggesting that power might be relatively poor. In addition, all methods based on means can have relatively low power under arbitrarily small departures from normality. Currently, a method based on 20% trimmed means and a percentile bootstrap method performs relatively well (Wilcox, in press). However, symmetric trimming was used, even when sampling from a highly skewed distribution and a rigid adherence to 20% trimming can result in low efficiency when a distribution is sufficiently heavy-tailed. Robust M-estimators are more flexible but they can be unsatisfactory in terms of Type I errors when sample sizes are small. This paper describes an alternative approach based on a modified one-step M-estimator that introduces more flexibility than a trimmed mean but provides better control over Type I error probabilities compared with using a one-step M-estimator.     

Robust analysis of variance

SUMMARY For the c -sample location problem with equal and unequal variances, we compare the classical F -test and its robustified version-the Welch test-with some nonparametric counterparts defined for two-sided and one-sided ordered alternatives, such as trend and umbrella alternatives. A new rank test for long-tailed distributions is proposed. The comparison is referred to level alpha and power beta of the tests, and is carried out via Monte Carlo simulation, assuming short-, medium- and long-tailed as well as asymmetric distributions. It turns out that the Welch test is the best one in the case of unequal variances but in the case of equal variances special non-parametric tests are to prefer.     

Levels of significance of some two-sample tests wkem observations are from compound normal distributions

This paper presents an investigation of the behavior of the levels of significance of the two-sample t and its related tests and the Mann-Whitney test when the samples are randomly drawn from mixtures of two normal populations (compound normals) and when the sample sizes are small (combined sample sizes ? 15). The use of the compound normal allows for investigation when the underlying populations are unequal, nonnormal, heterogeneous in variances, unimodal or bimodal, possessing smaller than normal kurtosis or containing contamination. The exact distribution of the t and its related tests are given. However, they are not readily amenable to calculations. Most of the numerical results presented were obtained by simulations     

Contributions to two-sample statistics

When testing the equality of the means from two independent normally distributed populations given that the variances of the two populations are unknown but assumed equal, the classical Student's two-sample t-test is recommended. If the underlying population distributions are normal with unequal and unknown variances, either Welch's t-statistic or Satterthwaite's approximate F test is suggested. However, Welch's procedure is non-robust under most non-normal distributions. There is a variable tolerance level around the strict assumptions of data independence, homogeneity of variances, and identical and normal distributions. Few textbooks offer alternatives when one or more of the underlying assumptions are not defensible. While there are more than a few non-parametric (rank) procedures that provide alternatives to Student's t-test, we restrict this review to the promising alternatives to Student's two-sample t-test in non-normal models.     

Application of Anbar's Approach to Hypothesis Testing to Detect the Difference between Two Proportions

In this paper, Anbar's (1983) approach for estimating a difference between two binomial proportions is discussed with respect to a hypothesis testing problem. Such an approach results in two possible testing strategies. While the results of the tests are expected to agree for a large sample size when two proportions are equal, the tests are shown to perform quite differently in terms of their probabilities of a Type I error for selected sample sizes. Moreover, the tests can lead to different conclusions, which is illustrated via a simple example; and the probability of such cases can be relatively large. In an attempt to improve the tests while preserving their relative simplicity feature, a modified test is proposed. The performance of this test and a conventional test based on normal approximation is assessed. It is shown that the modified Anbar's test better controls the probability of a Type I error for moderate sample sizes.