Distribution of variance ratio in unbalanced random model under heterogeneity of error variances

The effects of heteroscedasticity have been studied on the mean and variance of F ratio and on the power of F-test in unbalanced one-way random model, numerically. The computed results reveal that the heteroscedasticity and unbalanoedness have combined effects. The mean and variance of F as well as the power of F-test increase with inequality of error variances under balanced and those unbalanced situations where more variable groups have larger size. The effects are of serious nature when more variable groups have smaller size.     

Deletion residuals in the detection of heterogeneity of variances in linear regression

The heterogeneity of error variance often causes a huge interpretive problem in linear regression analysis. Before taking any remedial measures we first need to detect this problem. A large number of diagnostic plots are now available in the literature for detecting heteroscedasticity of error variances. Among them the ‘residuals’ and ‘fits’ (R–F) plot is very popular and commonly used. In the R–F plot residuals are plotted against the fitted responses, where both these components are obtained using the ordinary least squares (OLS) method. It is now evident that the OLS fits and residuals suffer a huge setback in the presence of unusual observations and hence the R–F plot may not exhibit the real scenario. The deletion residuals based on a data set free from all unusual cases should estimate the true errors in a better way than the OLS residuals. In this paper we propose ‘deletion residuals’ and the ‘deletion fits’ (DR–DF) plot for the detection of the heterogeneity of error variances in a linear regression model to get a more convincing and reliable graphical display. Examples show that this plot locates unusual observations more clearly than the R–F plot. The advantage of using deletion residuals in the detection of heteroscedasticity of error variance is investigated through Monte Carlo simulations under a variety of situations.     

Comparison of nonparametric analysis of variance methods: A vote for van der Waerden

ABSTRACT

For two-way layouts in a between-subjects analysis of variance design, the parametric F-test is compared with seven nonparametric methods: rank transform (RT), inverse normal transform (INT), aligned rank transform (ART), a combination of ART and INT, Puri & Sen's L statistic, Van der Waerden, and Akritas and Brunners ANOVA-type statistics (ATS). The type I error rates and the power are computed for 16 normal and nonnormal distributions, with and without homogeneity of variances, for balanced and unbalanced designs as well as for several models including the null and the full model. The aim of this study is to identify a method that is applicable without too much testing for all the attributes of the plot. The Van der Waerden test shows the overall best performance though there are some situations in which it is disappointing. The Puri & Sen's and the ATS tests show generally very low power. These two and the other methods cannot keep the type I error rate under control in too many situations. Especially in the case of lognormal distributions, the use of any of the rank-based procedures can be dangerous for cell sizes above 10. As already shown by many other authors, nonnormal distributions do not violate the parametric F-test, but unequal variances do, and heterogeneity of variances leads to an inflated error rate more or less also for the nonparametric methods. Finally, it should be noted that some procedures show rising error rates with increasing cell sizes, the ART, especially for discrete variables, and the RT, Puri & Sen, and the ATS in the cases of heteroscedasticity.     

Anomalies in the analysis of calibrated data

This study examines the effects of calibration errors on model assumptions and data-analytic tools in direct calibration assays. These effects encompass induced dependencies, inflated variances, and heteroscedasticity among the calibrated measurements, whose distributions arise as mixtures. These anomalies adversely affect conventional inferences, including the inconsistency of sample means; the underestimation of measurement variance; and the distributions of sample means, sample variances, and student's t as mixtures. Inferences in comparative experiments remain largely intact, although error mean squares continue to underestimate the measurement variances. These anomalies are masked in practice, as conventional diagnostics cannot discern the irregularities induced through calibration. Case studies illustrate the principal issues.     

Efficient Estimation and Robust Inference of Linear Regression Models in the Presence of Heteroscedastic Errors and High Leverage Points

It is common for linear regression models that the error variances are not the same for all observations and there are some high leverage data points. In such situations, the available literature advocates the use of heteroscedasticity consistent covariance matrix estimators (HCCME) for the testing of regression coefficients. Primarily, such estimators are based on the residuals derived from the ordinary least squares (OLS) estimator that itself can be seriously inefficient in the presence of heteroscedasticity. To get efficient estimation, many efficient estimators, namely the adaptive estimators are available but their performance has not been evaluated yet when the problem of heteroscedasticity is accompanied with the presence of high leverage data. In this article, the presence of high leverage data is taken into account to evaluate the performance of the adaptive estimator in terms of efficiency. Furthermore, our numerical work also evaluates the performance of the robust standard errors based on this efficient estimator in terms of interval estimation and null rejection rate (NRR).     

A Comparison of Bayesian Models of Heteroscedasticity in Nested Normal Data

We consider the fitting of a Bayesian model to grouped data in which observations are assumed normally distributed around group means that are themselves normally distributed, and consider several alternatives for accommodating the possibility of heteroscedasticity within the data. We consider the case where the underlying distribution of the variances is unknown, and investigate several candidate prior distributions for those variances. In each case, the parameters of the candidate priors (the hyperparameters) are themselves given uninformative priors (hyperpriors). The most mathematically convenient model for the group variances is to assign them inverse gamma distributed priors, the inverse gamma distribution being the conjugate prior distribution for the unknown variance of a normal population. We demonstrate that for a wide class of underlying distributions of the group variances, a model that assigns the variances an inverse gamma-distributed prior displays favorable goodness-of-fit properties relative to other candidate priors, and hence may be used as standard for modeling such data. This allows us to take advantage of the elegant mathematical property of prior conjugacy in a wide variety of contexts without compromising model fitness. We test our findings on nine real world publicly available datasets from different domains, and on a wide range of artificially generated datasets.     

New monte carlo results on the robustness of the anova f,w and f statistics

Because the usual F test for equal means is not robust to unequal variances, Brown and Forsythe (1974a) suggest replacing F with the statistics F or W which are based on the Satterthwaite and Welch adjusted degrees of freedom procedures. This paper reports practical situations where both F and W give * unsatisfactory results. In particular, both F and W may not provide adequate control over Type I errors. Moreover, for equal variances, but unequal sample sizes, W should be avoided in favor of F (or F ), but for equal sample sizes, and possibly unequal variances, W was the only satisfactory statistic. New results on power are included as well. The paper also considers the effect of using F or W only after a significant test for equal variances has been obtained, and new results on the robustness of the F test are described. It is found that even for equal sample sizes as large as 50 per treatment group, there are practical situations where the F test does not provide adequately control over the probability of a Type I error.     

Effects of non-normality and mild heteroscedasticity on estimators in regression

Several estimators are examined for the simple linear regression model under a controlled, experimental situation with multiple observations at each design point. The model is examined under normal and non-normal error distributions and mild heterogeneity of variances across the chosen design points. We consider the ordinary, generalized, and estimated generalized least squares estimators and several examples of M estimators. The asymptotic properties of the M estimator using the Huber ψ are presented under these conditions for the multiple regression model. A simulation study is also presented which indicates that the M estimator possesses strong robustness properties under the presence of both non-normality and mild heteroscedasticity o£ errors. Finally, the M estimates are compared to the least squares estimates in two examples.     

Statistical inference for a semiparametric measurement error regression model with heteroscedastic errors

Efficient inference for regression models requires that the heteroscedasticity be taken into account. We consider statistical inference under heteroscedasticity in a semiparametric measurement error regression model, in which some covariates are measured with errors. This paper has multiple components. First, we propose a new method for testing the heteroscedasticity. The advantages of the proposed method over the existing ones are that it does not need any nonparametric estimation and does not involve any mismeasured variables. Second, we propose a new two-step estimator for the error variances if there is heteroscedasticity. Finally, we propose a weighted estimating equation-based estimator (WEEBE) for the regression coefficients and establish its asymptotic properties. Compared with existing estimators, the proposed WEEBE is asymptotically more efficient, avoids undersmoothing the regressor functions and requires less restrictions on the observed regressors. Simulation studies show that the proposed test procedure and estimators have nice finite sample performance. A real data set is used to illustrate the utility of our proposed methods.     

LOGISTIC REGRESSION COMPARED TO NORMAL DISCRIMINATION FOR NON-NORMAL POPULATIONS'

An investigation is undertaken of the logistic regression procedure for estimating the posterior probability of an object belonging to one of two populations. The asymptotic bias and mean square error associated with the procedure are derived for univariate populations whose distributions satisfy the general Day-Kerridge model for which the logistic form is valid for the posterior probability. These properties are compared with those of the normal discrimination method based on the classical assumption of normal populations with common variances. The asymptotic relative efficiency of logistic regression is considered on the basis of asymptotic mean square error.     

Small area estimation via heteroscedastic nested‐error regression

We show that the maximum likelihood estimators (MLEs) of the fixed effects and within‐cluster correlation are consistent in a heteroscedastic nested‐error regression (HNER) model with completely unknown within‐cluster variances under mild conditions. The result implies that the empirical best linear unbiased prediction (EBLUP) method for small area estimation is valid in such a case. We also show that ignoring the heteroscedasticity can lead to inconsistent estimation of the within‐cluster correlation and inferior predictive performance. A jackknife measure of uncertainty for the EBLUP is developed under the HNER model. Simulation studies are carried out to investigate the finite‐sample performance of the EBLUP and MLE under the HNER model, with comparisons to those under the nested‐error regression model in various situations, as well as that of the jackknife measure of uncertainty. The well‐known Iowa crops data is used for illustration. The Canadian Journal of Statistics 40: 588–603; 2012 © 2012 Statistical Society of Canada     

Empirical distribution function under heteroscedasticity

Neglecting heteroscedasticity of error terms may imply the wrong identification of a regression model (see appendix). Employment of (heteroscedasticity resistent) White's estimator of covariance matrix of estimates of regression coefficients may lead to the correct decision about the significance of individual explanatory variables under heteroscedasticity. However, White's estimator of covariance matrix was established for least squares (LS)-regression analysis (in the case when error terms are normally distributed, LS- and maximum likelihood (ML)-analysis coincide and hence then White's estimate of covariance matrix is available for ML-regression analysis, tool). To establish White's-type estimate for another estimator of regression coefficients requires Bahadur representation of the estimator in question, under heteroscedasticity of error terms. The derivation of Bahadur representation for other (robust) estimators requires some tools. As the key too proved to be a tight approximation of the empirical distribution function (d.f.) of residuals by the theoretical d.f. of the error terms of the regression model. We need the approximation to be uniform in the argument of d.f. as well as in regression coefficients. The present paper offers this approximation for the situation when the error terms are heteroscedastic.     

Heteroscedasticity in non-stationary time series,some Monte Carlo evidence

A frequent question raised by practitioners doing unit root tests is whether these tests are sensitive to the presence of heteroscedasticity. Theoretically this is not the case for a wide range of heteroscedastic models. However, for some limiting cases such as degenerate and integrated heteroscedastic processes it is not obvious whether this will have an effect. In this paper we report a Monte Carlo study analyzing the implications of various types of heteroscedasticity on three types of unit root tests: The usual Dickey-Fuller test, Phillips' (1987) semi-parametric test and finally a Dickey-Fuller type test using White's (1980) heteroscedasticity consistent standard errors. The sorts of heteroscedasticity we examine are the GARCH model of Bollerslev (1986) and the Exponential ARCH model of Nelson (1991). In particular, we call attention to situations where the conditional variances exhibit a high degree of persistence as is frequently observed for returns of financial time series, and the case where, in fact, the variance process for the first class of models becomes degenerate.     

A non-parametric maximum test for the Behrens–Fisher problem

Non-normality and heteroscedasticity are common in applications. For the comparison of two samples in the non-parametric Behrens–Fisher problem, different tests have been proposed, but no single test can be recommended for all situations. Here, we propose combining two tests, the Welch t test based on ranks and the Brunner–Munzel test, within a maximum test. Simulation studies indicate that this maximum test, performed as a permutation test, controls the type I error rate and stabilizes the power. That is, it has good power characteristics for a variety of distributions, and also for unbalanced sample sizes. Compared to the single tests, the maximum test shows acceptable type I error control.     

New polya and inverse polya distributions of order k

New Polya and inverse Polya distributions of order k are derived by means of generalized urn models and by compounding the binomial and negative binomial distributions of order k of Philippou (1986, 1983) with the beta distribution. It i s noted that the present Polpa distribution of order k includes as special cases a new hypergeometric distribution of order k, a negative one,an inverse one, and a discrete uniform of the same order. The probability generating functions, means and variances of the new distributions are obtained, and five asymptotic results are established relating them to the abovedmentioned binomial and negative binomial distributions of order k, and to the Poisson distribution of the same order of Philippou (1983).Moment estimates are also given and applications are indicated.     

A monte carlo of plotting positions

A Monte Carlo study was made of the effects of using simple linear regression, on the appropriate probability paper, to estimate parameters, quantiles and cumulative probability for several distributions. These distributions were the Normal, Weibull (shape parameters 1, 2, and 4) and the Type I largest extreme-value distributions. The specific objective was to observe differences arising from choice of plotting positions. Plotting positions used were i/(n+l), (i?3)/(n+.04), (i?.5)/n, either (i?.375)/(n+.25) or (i?.4)/(n+.2), and either F[E(Y_i)] or F[E(£n Y)]. For each combination of 4 sample sizes (n=10(10)(40)), distribution, and plotting position, regression lines were found for each of N =9999 samples. Each regression line was used to estimate: (1) quantiles of 9 specific probabilities, (2) probabilities of 9 specific quantiles, and (3) return periods corresponding to 9 specific quantiles. Compa[rgrave]ison of the mean, variances, mean square error and medians of these estimates and of the regression coefficients confirm some results of Harter [Commun. Statist. A13(13), 1984] and provide further insight.     

Multiobjective Stochastic Multivariate Stratified Sampling in Presence of Nonresponse

The case of nonresponse in multivariate stratified sampling survey was first introduced by Hansen and Hurwitz in 1946 considering the sampling variances and costs to be deterministic. However, in real life situations sampling variance and cost are often random (stochastic) and have probability distributions. In this article, we have formulated the multivariate stratified sampling in the presence of nonresponse with random sampling variances and costs as a multiobjective stochastic programming problem. Here, the sampling variance and costs are considered random and converted into a deterministic NLPP by using chance constraint and modified E-model. A solution procedure using three different approaches are adopted viz. goal programming, fuzzy programming, and D₁ distance method to obtain the compromise allocation for the formulated problem. An empirical study has also been provided to illustrate the computational details.     

Testing equality of regression coefficients in heteroscedastic normal regression models

This article addresses the problem of testing whether the vectors of regression coefficients are equal for two independent normal regression models when the error variances are unknown. This problem poses severe difficulties both to the frequentist and Bayesian approaches to statistical inference. In the former approach, normal hypothesis testing theory does not apply because of the unrelated variances. In the latter, the prior distributions typically used for the parameters are improper and hence the Bayes factor-based solution cannot be used.We propose a Bayesian solution to this problem in which no subjective input is considered. We first generate “objective” proper prior distributions (intrinsic priors) for which the Bayes factor and model posterior probabilities are well defined. The posterior probability of each model is used as a model selection tool. This consistent procedure of testing hypotheses is compared with some of the frequentist approximate tests proposed in the literature.     

Properties of maximum likelihood estimators of variance components in the one-way classification,balanced data

We studied properties of maximum likelihood estimators (MLEs) of the variance components obtained from balanced data of the one-way classification. Exact and asymptotic expected values and variances of these MLEs were derived under the usual normality assumptions. Numerical studies illustrate these expected values and variances, and also illustrate the probability of obtaining a negative solution to the maximum likelihood (ML) equation for the between-class variance component. Simulations were used to study the robustness of the ML estimators under non-normal distributions.     

A new computational approach test for one-way ANOVA under heteroscedasticity

This paper proposes a new test statistic based on the computational approach test (CAT) for one-way analysis of variance (ANOVA) under heteroscedasticity. The proposed test was compared with other popular tests according to type I error and power of tests under different combinations of variances, means, number of groups and sample sizes. As a result, it was observed that the proposed test yields better results than other tests in many cases.