Impacts of auxiliary information on outliers in regression

Iheil and Goldberger (1961) and Theil (1963) founded the mixed regression approach, Their mixed regression estimator is essentially a large class of estimators that includes ridge, generalized ridge and shrinkage estimators, Properties of these estimators when data contain outliers have not been examined extensively. The present investigation shows that the mixed regression estimator, when observationsare subject to shift in means and variances, is uniformly superior, in terms of squared bias and variance, to the least squares estimator.     

Serial correlation or random subject effects

In longitudinal data analysis with random subject effects, there is often within subject serial correlation and possibly unequally spaced observations. This serial correlation can be partially confounded with the random between subject effects. In real data, it is often not clear whether there is serial correlation, random subject effects or both. Using inference based on the likelihood function, it is not always possible to identify the correct model, especially in small samples. However, it is important that some effort be made to attempt to find a good model rather than just making assumptions. This often means trying models with random coefficients, with serial correlation, and with both. Model selection criteria such as likelihood ratio tests and Akaike's Information Criterion (AIC) can be used. The problem of modelling serial correlation with unequally spaced observations is addressed. A real data example is presented where there is an apparent heterogeneity of variances, possible serial correlation and between subject random effects. In this example, it turns out that the random subject effects explains both the serial correlation and the variance heterogeneity.     

Two non-parametric, analysis-of-means-type tests for homogeneity of variances

After a brief review of the literature, two non-parametric tests for homogeneity of variances are presented. The first test is based on the analysis of means for ranks, which is a non-parametric version of the analysis of means (ANOM) that uses ranks as input for an ANOM test. The second test uses inverse normal scores of the ranks of scale transformations of the observations as input to the ANOM. Both homogeneity of variances tests can be presented in a graphical form, which makes it easy for practitioners to assess the practical and the statistical significance. A Monte Carlo study is used to show that these tests have power comparable with that of well-known robust tests for homogeneity of variances.     

Asymptotic variances of maximum likelihood estimator for the correlation coefficient from a BVN distribution with one variable subject to censoring

This paper deals with the problem of estimating the Pearson correlation coefficient when one variable is subject to left or right censoring. In parallel to the classical results on the Pearson correlation coefficient, we derive a workable formula, through tedious computation and intensive simplification, of the asymptotic variances of the maximum likelihood estimators in two cases: (1) known means and variances and (2) unknown means and variances. We illustrate the usefulness of the asymptotic results in experimental designs.     

Designs to guard against outliers in the présence or absence of model bias

Optimality of experimental design is considered in the situation in which individual observations may be subject to a shift in mean. The criterion of minimum average integrated mean square error is examined in general and the consequences for first and second degree models in the présence or absence of model bias are discussed.     

INFLUENCE FUNCTIONS IN FUNCTIONAL MEASUREMENT ERROR MODELS WITH REPLICATED DATA

We consider the construction and properties of influence functions in the context of functional measurement error models with replicated data. In these models estimates of the parameters can be affected both by the individual observations and the means of replicated observations. We show that influence function of the means of replicates on the estimate of regression coefficients can be only derived under the assumption that the variances of the errors are known, while one for the individual observations can be only derived simultaneously with their influence function on the estimators of the variances of the errors.     

The effect of conditional heteroskedasticity on common statistical procedures for means and variances

Summary: Commonly used standard statistical procedures for means and variances (such as the t–test for means or the F–test for variances and related confidence procedures) require observations from independent and identically normally distributed variables. These procedures are often routinely applied to financial data, such as asset or currency returns, which do not share these properties. Instead, they are nonnormal and show conditional heteroskedasticity, hence they are dependent. We investigate the effect of conditional heteroskedasticity (as modelled by GARCH(1,1)) on the level of these tests and the coverage probability of the related confidence procedures. It can be seen that conditional heteroskedasticity has no effect on procedures for means (at least in large samples). There is, however, a strong effect of conditional heteroskedasticity on procedures for variances. These procedures should therefore not be used if conditional heteroskedasticity is prevalent in the data.*We are grateful to the referees for their useful and constructive comments.     

The performance of tests when observations have different variances

It is common to test if there is an effect due to a treatment. The commonly used tests have the assumption that the observations differ in location, and that their variances are the same over the groups. Different variances can arise if the observations being analyzed are means of different numbers of observations on individuals or slopes of growth curves with missing data. This study is concerned with cases in which the unequal variances are known, or known to a constant of proportionality. It examines the performance of the ttest, the Mann–Whitney–Wilcoxon Rank Sum test, the Median test, and the Van der Waerden test under these conditions. The t-test based on the weighted means is the likelihood ratio test under normality and has the usual optimality properties. The other tests are compared to it. One may align and scale the observations by subtracting the mean and dividing by the standard deviation of each point. This leads to other, analogous test statistics based on these adjusted observations. These statistics are also compared. Finally, the regression scores tests are compared to the other procedures.     

Improved maximum likelihood estimators in a heteroskedastic errors-in-variables model

This paper develops a bias correction scheme for a multivariate heteroskedastic errors-in-variables model. The applicability of this model is justified in areas such as astrophysics, epidemiology and analytical chemistry, where the variables are subject to measurement errors and the variances vary with the observations. We conduct Monte Carlo simulations to investigate the performance of the corrected estimators. The numerical results show that the bias correction scheme yields nearly unbiased estimates. We also give an application to a real data set.     

Comparison of Three Sampling Designs

Three sampling designs are considered for estimating the sum of k population means by the sum of the corresponding sample means. These are (a) the optimal design; (b) equal sample sizes from all populations; and (c) sample sizes that render equal variances to all sample means. Designs (b) and (c) are equally inefficient, and may yield a variance up to k times as large as that of (a). Similar results are true when the cost of sampling is introduced, and they depend on the population sampled.     

Robustness of Optimal Chemical Balance Weighing Designs for Estimation of Total Weight

The subject of our study is the chemical balance weighing design in which the errors are correlated and have equal variances. A lower bound for the variance of estimated total weight is obtained and the necessary and sufficient conditions for the attainability of this lower bound are given. Some designs for which the lower bound is attainable are considered.     

A Comparison of Bayesian Models of Heteroscedasticity in Nested Normal Data

We consider the fitting of a Bayesian model to grouped data in which observations are assumed normally distributed around group means that are themselves normally distributed, and consider several alternatives for accommodating the possibility of heteroscedasticity within the data. We consider the case where the underlying distribution of the variances is unknown, and investigate several candidate prior distributions for those variances. In each case, the parameters of the candidate priors (the hyperparameters) are themselves given uninformative priors (hyperpriors). The most mathematically convenient model for the group variances is to assign them inverse gamma distributed priors, the inverse gamma distribution being the conjugate prior distribution for the unknown variance of a normal population. We demonstrate that for a wide class of underlying distributions of the group variances, a model that assigns the variances an inverse gamma-distributed prior displays favorable goodness-of-fit properties relative to other candidate priors, and hence may be used as standard for modeling such data. This allows us to take advantage of the elegant mathematical property of prior conjugacy in a wide variety of contexts without compromising model fitness. We test our findings on nine real world publicly available datasets from different domains, and on a wide range of artificially generated datasets.     

Detection and Interpretation of a Multivariate Signal Using Combined Charts

A control procedure is presented in this article that is based on jointly using two separate control statistics in the detection and interpretation of signals in a multivariate normal process. The procedure detects the following three situations: (i) a mean vector shift without a shift in the covariance matrix; (ii) a shift in process variation (covariance matrix) without a mean vector shift; and (iii) both a simultaneous shift in the mean vector and covariance matrix as the result of a change in the parameters of some key process variables. It is shown that, following the occurrence of a signal on either of the separate control charts, the values from both of the corresponding signaling statistics can be decomposed into interpretable elements. Viewing the two decompositions together helps one to specifically identify the individual components and associated variables that are being affected. These components may include individual means or variances of the process variables as well as the correlations between or among variables. An industrial data set is used to illustrate the procedure.     

Optimal cross-over designs for nonlinear mixed models using a first-order expansion

We consider the construction of optimal cross-over designs for nonlinear mixed effect models based on the first-order expansion. We show that for AB/BA designs a balanced subject allocation is optimal when the parameters depend on treatments only. For multiple period, multiple sequence designs, uniform designs are optimal among dual balanced designs under the same conditions. As a by-product, the same results hold for multivariate linear mixed models with variances depending on treatments.     

Semiparametric mixture models and repeated measures: the multinomial cut point model

Summary.  Suppose that we have m repeated measures on each subject, and we model the observation vectors with a finite mixture model.  We further assume that the repeated measures are conditionally independent. We present methods to estimate the shape of the component distributions along with various features of the component distributions such as the medians, means and variances. We make no distributional assumptions on the components; indeed, we allow different shapes for different components.     

Bayesian Estimators for Small Area Models Shrinking Both Means and Variances

For small area estimation of area‐level data, the Fay–Herriot model is extensively used as a model‐based method. In the Fay–Herriot model, it is conventionally assumed that the sampling variances are known, whereas estimators of sampling variances are used in practice. Thus, the settings of knowing sampling variances are unrealistic, and several methods are proposed to overcome this problem. In this paper, we assume the situation where the direct estimators of the sampling variances are available as well as the sample means. Using this information, we propose a Bayesian yet objective method producing shrinkage estimation of both means and variances in the Fay–Herriot model. We consider the hierarchical structure for the sampling variances, and we set uniform prior on model parameters to keep objectivity of the proposed model. For validity of the posterior inference, we show under mild conditions that the posterior distribution is proper and has finite variances. We investigate the numerical performance through simulation and empirical studies.     

A monte carlo comparison of the smoothing,scoring and em algorithms for dispersion matrix estimation with incomplete growth curve data

Incomplete growth curve data often result from missing or mistimed observations in a repeated measures design. Virtually all methods of analysis rely on the dispersion matrix estimates. A Monte Carlo simulation was used to compare three methods of estimation of dispersion matrices for incomplete growth curve data. The three methods were: 1) maximum likelihood estimation with a smoothing algorithm, which finds the closest positive semidefinite estimate of the pairwise estimated dispersion matrix; 2) a mixed effects model using the EM (estimation maximization) algorithm; and 3) a mixed effects model with the scoring algorithm. The simulation included 5 dispersion structures, 20 or 40 subjects with 4 or 8 observations per subject and 10 or 30% missing data. In all the simulations, the smoothing algorithm was the poorest estimator of the dispersion matrix. In most cases, there were no significant differences between the scoring and EM algorithms. The EM algorithm tended to be better than the scoring algorithm when the variances of the random effects were close to zero, especially for the simulations with 4 observations per subject and two random effects.     

Estimation of two order restricted normal means with unknown and possibly unequal variances

Estimation of each of and linear functions of two order restricted normal means is considered when variances are unknown and possibly unequal. We replace unknown variances with sample variances and construct isotonic regression estimators, which we call in our paper the plug-in estimators, to estimate ordered normal means. Under squared error loss, a necessary and sufficient condition is given for the plug-in estimators to improve upon the unrestricted maximum likelihood estimators uniformly. As for the estimation of linear functions of ordered normal means, we also show that when variances are known, the restricted maximum likelihood estimator always improves upon the unrestricted maximum likelihood estimator uniformly, but when variances are unknown, the plug-in estimator does not always improve upon the unrestricted maximum likelihood estimator uniformly.     

On the comparison of Fisher information of the Weibull and GE distributions

In this paper, we consider the Fisher information matrices of the generalized exponential (GE) and Weibull distributions for complete and Type-I censored observations. Fisher information matrix can be used to compute asymptotic variances of the different estimators. Although both distributions may provide similar data fit but the corresponding Fisher information matrices can be quite different. Moreover, the percentage loss of information due to truncation of the Weibull distribution is much more than the GE distribution. We compute the total information of the Weibull and GE distributions for different parameter ranges. We compare the asymptotic variances of the median estimators and the average asymptotic variances of all the percentile estimators for complete and Type-I censored observations. One data analysis has been preformed for illustrative purposes. When two fitted distributions are very close to each other and very difficult to discriminate otherwise, the Fisher information or the above mentioned asymptotic variances may be used for discrimination purposes.     

Robust statistics for testing equality of means or variances

For testing the equality of means (location parameters) of two populations, Tiku (1980a) defined a statistic Tc (based on symmetrically censored samples) and showed that this statistic is robust to underlying populations and is also remarkably powerful. In this paper, we define a similar statistic T (based on samples s with observations censored only on one side) and show that this stat is tic is more powerful than T and nonparametric statistics, C for skew populations. We also provide a modification of this statistic for testing the equality of two population variances.