Stein's two sample test of a general linear hypothesis and a noncentral f–type distribution

Stein's two–sample procedure for a general linear model is studied and derived in terms of matrices in which the error tems are distributed as multivatriate student t–error terms. Tests and confidence regions are constructed in a similar way to classical linear models which involves percentage points of student t and F distributions. The advantages of taking two samples are: the variance of the error terms is known, and the power of tests are size of confidence regions are controllable. A new distribution called noncentral F–type distribution different from the nencentral F is found when considerinf the power of the test of general linear hypothesis.     

Comparing the overlapping of two independent confidence intervals with a single confidence interval for two normal population parameters

Two overlapping confidence intervals have been used in the past to conduct statistical inferences about two population means and proportions. Several authors have examined the shortcomings of Overlap procedure and have determined that such a method distorts the significance level of testing the null hypothesis of two population means and reduces the statistical power of the test. Nearly all results for small samples in Overlap literature have been obtained either by simulation or by formulas that may need refinement for small sample sizes, but accurate large sample information exists. Nevertheless, there are aspects of Overlap that have not been presented and compared against the standard statistical procedure. This article will present exact formulas for the maximum % overlap of two independent confidence intervals below which the null hypothesis of equality of two normal population means or variances must still be rejected for any sample sizes. Further, the impact of Overlap on the power of testing the null hypothesis of equality of two normal variances will be assessed. Finally, the noncentral t-distribution is used to assess the Overlap impact on type II error probability when testing equality of means for sample sizes larger than 1.     

Exact tests and confidence regions in nonlinear regression

Nonlinear regression models with spherically symmetric error vectors and a single nonlinear parameter are considered. On the basis of a new geometric approach, exact one- and two-sided tests and confidence regions for the nonlinear parameter are derived in the cases of known and unknown error variances. A geometric measure representation formula is used to determine the power functions of the tests if the error variance is known and to derive different lower bounds for the power function of a one-sided test in the case of an unknown error variance. The latter can be done quite effectively by constructing and measuring several balls inside the critical region. A numerical study compares the results for different density generating functions of the error distribution.     

Testing equality of population variances the robust way

The purpose of this paper is to investigate the robustness (stability of Type I error to deviations from normality) and power properties of various tests for testing equality of population variances. It is shown that the tests based on Tiku’ s (1967, 1980, 1982) MML estimators have good robustness properties and are the most powerful overall.     

Testing equality of variances for several normal populations

In this study, we develop a test based on computational approach for the equality of variances of several normal populations. The proposed method is numerically compared with the existing methods. The numeric results demonstrate that the proposed method performs very well in terms of type I error rate and power of test. Furthermore we study the robustness of the tests by using simulation study when the underlying data are from t, exponential and uniform distributions. Finally we analyze a real dataset that motivated our study using the proposed test.     

Effect size for comparing two or more normal distributions based on maximal contrasts in outcomes

Effect size is a concept that can be especially useful in bioequivalence and studies designed to find important and not just statistically significant differences among responses to treatments based on independent random samples. We develop and explore a new effect size related to a maximal superiority ordering for assessing the separation among two or more normal distributions, possibly having different means and different variances. Confidence intervals and tests of hypothesis for this effect size are developed using a p value obtained by averaging over a distribution on variances. Since there is almost always some difference among treatments, instead of the usual hypothesis test of exactly no effect, researchers should consider testing that an appropriate effect size has at least, or at most, some meaningful magnitude, when one is available, possibly established using the framework developed here. A simulation study of type I error rate, power and interval length is presented. R-code for constructing the confidence intervals and carrying out the tests here can be downloaded from Author’s website.     

Two separate effects of variance heterogeneity on the validity and power of significance tests of location

Heterogeneity of variances of treatment groups influences the validity and power of significance tests of location in two distinct ways. First, if sample sizes are unequal, the Type I error rate and power are depressed if a larger variance is associated with a larger sample size, and elevated if a larger variance is associated with a smaller sample size. This well-established effect, which occurs in t and F tests, and to a lesser degree in nonparametric rank tests, results from unequal contributions of pooled estimates of error variance in the computation of test statistics. It is observed in samples from normal distributions, as well as non-normal distributions of various shapes. Second, transformation of scores from skewed distributions with unequal variances to ranks produces differences in the means of the ranks assigned to the respective groups, even if the means of the initial groups are equal, and a subsequent inflation of Type I error rates and power. This effect occurs for all sample sizes, equal and unequal. For the t test, the discrepancy diminishes, and for the Wilcoxon–Mann–Whitney test, it becomes larger, as sample size increases. The Welch separate-variance t test overcomes the first effect but not the second. Because of interaction of these separate effects, the validity and power of both parametric and nonparametric tests performed on samples of any size from unknown distributions with possibly unequal variances can be distorted in unpredictable ways.     

Comparison of confidence intervals for the Behrens-Fisher problem

The Behrens–Fisher problem concerns the inferences for the difference between means of two independent normal populations without the assumption of equality of variances. In this article, we compare three approximate confidence intervals and a generalized confidence interval for the Behrens–Fisher problem. We also show how to obtain simultaneous confidence intervals for the three population case (analysis of variance, ANOVA) by the Bonferroni correction factor. We conduct an extensive simulation study to evaluate these methods in respect to their type I error rate, power, expected confidence interval width, and coverage probability. Finally, the considered methods are applied to two real dataset.     

A Comparison of Analysis of Covariate-Adjusted Residuals and Analysis of Covariance

Various methods to control the influence of a covariate on a response variable are compared. These methods are ANOVA with or without homogeneity of variances (HOV) of errors and Kruskal–Wallis (K–W) tests on (covariate-adjusted) residuals and analysis of covariance (ANCOVA). Covariate-adjusted residuals are obtained from the overall regression line fit to the entire data set ignoring the treatment levels or factors. It is demonstrated that the methods on covariate-adjusted residuals are only appropriate when the regression lines are parallel and covariate means are equal for all treatments. Empirical size and power performance of the methods are compared by extensive Monte Carlo simulations. We manipulated the conditions such as assumptions of normality and HOV, sample size, and clustering of the covariates. The parametric methods on residuals and ANCOVA exhibited similar size and power when error terms have symmetric distributions with variances having the same functional form for each treatment, and covariates have uniform distributions within the same interval for each treatment. In such cases, parametric tests have higher power compared to the K–W test on residuals. When error terms have asymmetric distributions or have variances that are heterogeneous with different functional forms for each treatment, the tests are liberal with K–W test having higher power than others. The methods on covariate-adjusted residuals are severely affected by the clustering of the covariates relative to the treatment factors when covariate means are very different for treatments. For data clusters, ANCOVA method exhibits the appropriate level. However, such a clustering might suggest dependence between the covariates and the treatment factors, so makes ANCOVA less reliable as well.     

Generalized Method of Moments Estimation When a Parameter Is on a Boundary

This article establishes the asymptotic distributions of generalized method of moments (GMM) estimators when the true parameter lies on the boundary of the parameter space. The conditions allow the estimator objective function to be nonsmooth and to depend on preliminary estimators. The boundary of the parameter space may be curved and/or kinked. The article discusses three examples: (1) instrumental variables (IV) estimation of a regression model with nonlinear equality and/or inequality restrictions on the parameters; (2) method of simulated moments estimation of a multinomial discrete response model with some random coefficient variances equal to 0, some random effect variances equal to 0, or some measurement error variances equal to 0; and (3) semiparametric least squares estimation of a partially linear regression model with nonlinear equality and/or inequality restrictions on the parameters.     

Contrast analysis for scale differences

Research on tests for scale equality, that are robust to violations of the distributional normality assumption, have focused exclusively on an overall test statistic and have not examined procedures for identifying specific differences in multiple group designs. The present study compares four contrast analysis procedures for scale differences in the single factor four group design. Two data transformations are considered under several conbinations of variance difference, sample sizes, and distributional forms.The results indicate that no single transformation or analysis procedure is uniformly superior in controlling the familywise error rate or in statistical power. The relationship between sample size and variances is a major factor in selecting a contrast analysis procedure.     

Hypothesis Testing in Multivariate Linear Models with Randomly Missing Data

Tests for mean equality proposed by Weerahandi (1995) and Chen and Chen (1998), tests that do not require equality of population variances, were examined when data were not only heterogeneous but, as well, nonnormal in unbalanced completely randomized designs. Furthermore, these tests were compared to a test examined by Lix and Keselman (1998), a test that uses a heteroscedastic statistic (i.e., Welch, 1951) with robust estimators (20% trimmed means and Winsorized variances). Our findings confirmed previously published data that the tests are indeed robust to variance heterogeneity when the data are obtained from normal populations. However, the Weerahandi (1995) and Chen and Chen (1998) tests were not found to be robust when data were obtained from nonnormal populations. Indeed, rates of Type I error were typically in excess of 10% and, at times, exceeded 50%. On the other hand, the statistic presented by Lix and Keselman (1998) was generally robust to variance heterogeneity and nonnormality.     

A graphical method for testing the equality of several variances

SUMMARY The problem of testing the equality of several variances arises in many areas. For testing the equality of variances, several tests are available in the literature which demonstrate only the statistical significance of the variances. In this paper, a graphical method is presented for testing the equality of variances. This method simultaneously demonstrates the statistical and engineering significance. Two examples are given to illustrate the proposed graphical method, and the conclusions obtained are compared with the existing tests.     

Methods to test for equality of two normal distributions

Statistical tests for two independent samples under the assumption of normality are applied routinely by most practitioners of statistics. Likewise, presumably each introductory course in statistics treats some statistical procedures for two independent normal samples. Often, the classical two-sample model with equal variances is introduced, emphasizing that a test for equality of the expected values is a test for equality of both distributions as well, which is the actual goal. In a second step, usually the assumption of equal variances is discarded. The two-sample t test with Welch correction and the F test for equality of variances are introduced. The first test is solely treated as a test for the equality of central location, as well as the second as a test for the equality of scatter. Typically, there is no discussion if and to which extent testing for equality of the underlying normal distributions is possible, which is quite unsatisfactorily regarding the motivation and treatment of the situation with equal variances. It is the aim of this article to investigate the problem of testing for equality of two normal distributions, and to do so using knowledge and methods adequate to statistical practitioners as well as to students in an introductory statistics course. The power of the different tests discussed in the article is examined empirically. Finally, we apply the tests to several real data sets to illustrate their performance. In particular, we consider several data sets arising from intelligence tests since there is a large body of research supporting the existence of sex differences in mean scores or in variability in specific cognitive abilities.     

Comparison of ANOVA-F and ANOM tests with regard to type I error rate and test power

A Monte Carlo simulation was conducted to compare the type I error rate and test power of the analysis of means (ANOM) test to the one-way analysis of variance F-test (ANOVA-F). Simulation results showed that as long as the homogeneity of the variance assumption was satisfied, regardless of the shape of the distribution, number of group and the combination of observations, both ANOVA-F and ANOM test have displayed similar type I error rates. However, both tests have been negatively affected from the heterogeneity of the variances. This case became more obvious when the variance ratios increased. The test power values of both tests changed with respect to the effect size (Δ), variance ratio and sample size combinations. As long as the variances are homogeneous, ANOVA-F and ANOM test have similar powers except unbalanced cases. Under unbalanced conditions, the ANOVA-F was observed to be powerful than the ANOM-test. On the other hand, an increase in total number of observations caused the power values of ANOVA-F and ANOM test approach to each other. The relations between effect size (Δ) and the variance ratios affected the test power, especially when the sample sizes are not equal. As ANOVA-F has become to be superior in some of the experimental conditions being considered, ANOM is superior in the others. However, generally, when the populations with large mean have larger variances as well, ANOM test has been seen to be superior. On the other hand, when the populations with large mean have small variances, generally, ANOVA-F has observed to be superior. The situation became clearer when the number of the groups is 4 or 5.     

Simulating competing cointegration tests in a bivariate system

In this paper, we consider the size and power of a set of cointegration tests in a number of Monte Carlo simulations. The behaviour of the competing methods is investigated in diff erent situations, including diff erent levels of variance and correlation in the error processes. The impact of violations of the common factor restriction (CFR) implied by the Engle-Granger framework is studied in these situations. The reactions to changes in the CFR condition depend on the error correlation. When the correlation is non-positive, the power increases with increasing CFR violations for the error correction model (ECM) test, while the other tests react in the opposite direction. We also note the reaction to diff erences in the error variances in the data-generating process. For positive correlation and equal variances, the reaction to changes in the CFR violations diff ers somewhat between the tests. We conclude that the ECM and the Z-tests show the best performance over diff erent parameter combinations. In most situations the ECM is best. Therefore, if we had to recommend a unit root test, it would be the ECM, especially for small samples. However, we do not think that one should use just one test, but two or more. Of course, the portfolio of tests we have considered here only represents a subset of the possible tests.     

Estimation of the common location paeameters for the two linear models

Various estimators proposed for the estimation of a common mean are extended to the estimation of the common location parameters for two linear models including the estimators based on preliminary tests of equality of variances. Exact distribution of these estimates, simultaneous confidence bounds based on these estimates and the bounds on the variances of these estimates are obtained using different approaches.     

Robust tests for the equality of variances for clustered data

Tests for the equality of variances are often needed in applications. In genetic studies the assumption of equal variances of continuous traits, measured in identical and fraternal twins, is crucial for heritability analysis. To test the equality of variances of traits, which are non-normally distributed, Levene [H. Levene, Robust tests for equality of variances, in Contributions to Probability and Statistics, I. Olkin, ed. Stanford University Press, Palo Alto, California, 1960, pp. 278–292] suggested a method that was surprisingly robust under non-normality, and the procedure was further improved by Brown and Forsythe [M.B. Brown and A.B. Forsythe, Robust tests for the equality of variances, J. Amer. Statis. Assoc. 69 (1974), pp. 364–367]. These tests assumed independence of observations. However, twin data are clustered – observations within a twin pair may be dependent due to shared genes and environmental factors. Uncritical application of the tests of Brown and Forsythe to clustered data may result in much higher than nominal Type I error probabilities. To deal with clustering we developed an extended version of Levene's test, where the ANOVA step is replaced with a regression analysis followed by a Wald-type test based on a clustered version of the robust Huber–White sandwich estimator of the covariance matrix. We studied the properties of our procedure using simulated non-normal clustered data and obtained Type I error rates close to nominal as well as reasonable powers. We also applied our method to oral glucose tolerance test data obtained from a twin study of the metabolic syndrome and related components and compared the results with those produced by the traditional approaches.     

A revisit to test the equality of variances of several populations

We revisit the problem of testing homoscedasticity (or, equality of variances) of several normal populations which has applications in many statistical analyses, including design of experiments. The standard text books and widely used statistical packages propose a few popular tests including Bartlett's test, Levene's test and a few adjustments of the latter. Apparently, the popularity of these tests have been based on limited simulation study carried out a few decades ago. The traditional tests, including the classical likelihood ratio test (LRT), are asymptotic in nature, and hence do not perform well for small sample sizes. In this paper we propose a simple parametric bootstrap (PB) modification of the LRT, and compare it against the other popular tests as well as their PB versions in terms of size and power. Our comprehensive simulation study bursts some popularly held myths about the commonly used tests and sheds some new light on this important problem. Though most popular statistical software/packages suggest using Bartlette's test, Levene's test, or modified Levene's test among a few others, our extensive simulation study, carried out under both the normal model as well as several non-normal models clearly shows that a PB version of the modified Levene's test (which does not use the F-distribution cut-off point as its critical value), and Loh's exact test are the “best” performers in terms of overall size as well as power.     

An upper bound on a ratio of variances

Several processes may be monitored in terms of their variances relative to each other by the maximum ratio of variances. Based on the observed value an upper confidence bound for the population value is derived and tables are supplied for its implementation. Such bounds may be used when the experimenter wishes to establish that the variances are "not too different". The usual testing of variances for equality is noticed to be inappropriate in such cases.