The Gaussian rank correlation estimator: robustness properties

The Gaussian rank correlation equals the usual correlation coefficient computed from the normal scores of the data. Although its influence function is unbounded, it still has attractive robustness properties. In particular, its breakdown point is above 12%. Moreover, the estimator is consistent and asymptotically efficient at the normal distribution. The correlation matrix obtained from pairwise Gaussian rank correlations is always positive semidefinite, and very easy to compute, also in high dimensions. We compare the properties of the Gaussian rank correlation with the popular Kendall and Spearman correlation measures. A simulation study confirms the good efficiency and robustness properties of the Gaussian rank correlation. In the empirical application, we show how it can be used for multivariate outlier detection based on robust principal component analysis.     

Optimally robust estimators in generalized Pareto models

In this paper, we study the robustness properties of several procedures for the joint estimation of shape and scale in a generalized Pareto model. The estimators that we primarily focus upon, most bias robust estimator (MBRE) and optimal MSE-robust estimator (OMSE), are one-step estimators distinguished as optimally robust in the shrinking neighbourhood setting; that is, they minimize the maximal bias, respectively, on such a specific neighbourhood, the maximal mean squared error (MSE). For their initialization, we propose a particular location–dispersion estimator, MedkMAD, which matches the population median and kMAD (an asymmetric variant of the median of absolute deviations) against the empirical counterparts. These optimally robust estimators are compared to the maximum-likelihood, skipped maximum-likelihood, Cramér–von-Mises minimum distance, method-of-medians, and Pickands estimators. To quantify their deviation from robust optimality, for each of these suboptimal estimators, we determine the finite-sample breakdown point and the influence function, as well as the statistical accuracy measured by asymptotic bias, variance, and MSE – all evaluated uniformly on shrinking neighbourhoods. These asymptotic findings are complemented by an extensive simulation study to assess the finite-sample behaviour of the considered procedures. The applicability of the procedures and their stability against outliers are illustrated for the Danish fire insurance data set from the package evir.     

Double robust estimator in general treatment regimes based on Covariate-balancing

Double robust estimators have double the chance of being a consistent estimator of a causal effect in binary treatments cases. In this paper, we proposed an estimator of a causal effect for general treatment regimes based on covariate-balancing. Under parametrical situation, our estimator has double robustness.     

Influence functions applied to the estimation of mean rain rate

In this paper we illustrate the usefulness of influence functions for studying properties of various statistical estimators of mean rain rate using space-borne radar data. In Martin (1999), estimators using censoring, minimum chi-square, and least squares are compared in terms of asymptotic variance. Here, we use influence functions to consider robustness properties of the same estimators. We also obtain formulas for the asymptotic variance of the estimators using influence functions, and thus show that they may also be used for studying relative efficiency. The least squares estimator, although less efficient, is shown to be more robust in the sense that it has the smallest gross-error sensitivity. In some cases, influence functions associated with the estimators reveal counterintuitive behaviour. For example, observations that are less than the mean rain rate may increase the estimated mean. The additional information gleaned from influence functions may be used to understand better and improve the estimation procedures themselves.     

The influence function of penalized regression estimators

To perform regression analysis in high dimensions, lasso or ridge estimation are a common choice. However, it has been shown that these methods are not robust to outliers. Therefore, alternatives as penalized M-estimation or the sparse least trimmed squares (LTS) estimator have been proposed. The robustness of these regression methods can be measured with the influence function. It quantifies the effect of infinitesimal perturbations in the data. Furthermore, it can be used to compute the asymptotic variance and the mean-squared error (MSE). In this paper we compute the influence function, the asymptotic variance and the MSE for penalized M-estimators and the sparse LTS estimator. The asymptotic biasedness of the estimators make the calculations non-standard. We show that only M-estimators with a loss function with a bounded derivative are robust against regression outliers. In particular, the lasso has an unbounded influence function.     

Two-stage Huber estimation

In this paper we propose a new robust estimator in the context of two-stage estimation methods directed towards the correction of endogeneity problems in linear models. Our estimator is a combination of Huber estimators for each of the two stages, with scale corrections implemented using preliminary median absolute deviation estimators. In this way we obtain a two-stage estimation procedure that is an interesting compromise between concerns of simplicity of calculation, robustness and efficiency. This method compares well with other possible estimators such as two-stage least-squares (2SLS) and two-stage least-absolute-deviations (2SLAD), asymptotically and in finite samples. It is notably interesting to deal with contamination affecting more heavily the distribution tails than a few outliers and not losing as much efficiency as other popular estimators in that case, e.g. under normality. An additional originality resides in the fact that we deal with random regressors and asymmetric errors, which is not often the case in the literature on robust estimators.     

Minimum distance estimation in linear regression with strong mixing errors

Abstract

Minimum distance estimation on the linear regression model with independent errors is known to yield an efficient and robust estimator. We extend the method to the model with strong mixing errors and obtain an estimator of the vector of the regression parameters. The goal of this article is to demonstrate the proposed estimator still retains efficiency and robustness. To that end, this article investigates asymptotic distributional properties of the proposed estimator and compares it with other estimators. The efficiency and the robustness of the proposed estimator are empirically shown, and its superiority over the other estimators is established.     

Tukey’s M-estimator of the Poisson parameter with a special focus on small means

We treat robust M-estimators for independent and identically distributed Poisson data. We introduce modified Tukey M-estimators with bias correction and compare them to M-estimators based on the Huber function as well as to weighted likelihood and other estimators by simulation in case of clean data and data with outliers. In particular, we investigate the problem of combining robustness and high efficiencies at small Poisson means caused by the strong asymmetry of such Poisson distributions and propose a further estimator based on adaptive trimming. The advantages of the constructed estimators are illustrated by an application to smoothing count data with a time varying mean and level shifts.     

A two-step robust estimation of the process mean using M-estimator

Parameter estimation is the first step in constructing control charts. One of these parameters is the process mean. The classical estimators of the process mean are sensitive to the presence of outlying data and subgroups which contaminate the whole data. In existing robust estimators for the process mean, the effects of the presence of the individual outliers are being considered, while, in this paper, a robust estimator is being proposed to reduce the effect of outlying subgroups as well as the individual outliers within a subgroup. The proposed estimator was compared with some classical and robust estimators of the process mean. Although, its relative efficiency is fourth among the estimators tested, its robustness and efficiency are large when the outlying subgroups are present. Evaluation of the results indicated that the proposed estimator is less sensitive to the presence of outliers and the process mean performs well when there are no individual outliers or outlying subgroups.     

On minimum Hellinger distance estimation

Efficiency and robustness are two fundamental concepts in parametric estimation problems. It was long thought that there was an inherent contradiction between the aims of achieving robustness and efficiency; that is, a robust estimator could not be efficient and vice versa. It is now known that the minimum Hellinger distance approached introduced by Beran [R. Beran, Annals of Statistics 1977;5:445–463] is one way of reconciling the conflicting concepts of efficiency and robustness. For parametric models, it has been shown that minimum Hellinger estimators achieve efficiency at the model density and simultaneously have excellent robustness properties. In this article, we examine the application of this approach in two semiparametric models. In particular, we consider a two‐component mixture model and a two‐sample semiparametric model. In each case, we investigate minimum Hellinger distance estimators of finite‐dimensional Euclidean parameters of particular interest and study their basic asymptotic properties. Small sample properties of the proposed estimators are examined using a Monte Carlo study. The results can be extended to semiparametric models of general form as well. The Canadian Journal of Statistics 37: 514–533; 2009 © 2009 Statistical Society of Canada     

Robust ridge and robust Liu estimator for regression based on the LTS estimator

In the multiple linear regression analysis, the ridge regression estimator and the Liu estimator are often used to address multicollinearity. Besides multicollinearity, outliers are also a problem in the multiple linear regression analysis. We propose new biased estimators based on the least trimmed squares (LTS) ridge estimator and the LTS Liu estimator in the case of the presence of both outliers and multicollinearity. For this purpose, a simulation study is conducted in order to see the difference between the robust ridge estimator and the robust Liu estimator in terms of their effectiveness; the mean square error. In our simulations, the behavior of the new biased estimators is examined for types of outliers: X-space outlier, Y-space outlier, and X-and Y-space outlier. The results for a number of different illustrative cases are presented. This paper also provides the results for the robust ridge regression and robust Liu estimators based on a real-life data set combining the problem of multicollinearity and outliers.     

Minimum disparity estimation based on combined disparities: Asymptotic results

The maximum likelihood estimator (MLE) is asymptotically efficient for most parametric models under standard regularity conditions, but it has very poor robustness properties. On the other hand some of the minimum disparity estimators like the minimum Hellinger distance estimator (MHDE) have strong robustness features but their small sample efficiency at the model turns out to be very poor compared to the MLE. Methods based on the minimization of some combined disparities can substantially improve their small sample performances without affecting their robustness properties (Park et al., 1995). All studies involving the combined disparity have so far been empirical, and there are no results on the asymptotic properties of these estimators. In view of the usefulness of these procedures this is a major gap in theory, which we try to fill through the present work. Some illustrations of the performance of the estimators and the corresponding tests are also provided.     

Preliminary estimators for robust non-linear regression estimation

In this paper, the robustness of weighted non-linear least-squares estimation based on some preliminary estimators is examined. The preliminary estimators are the Lnorm estimates proposed by Schlossmacher, by El-Attar et al., by Koenker and Park, and by Lawrence and Arthur. A numerical example is presented to compare the robustness of the weighted non-linear least-squares approach when based on the preliminary estimators of Schlossmacher (HS), El-Attar et al. (HEA), Koenker and Park (HKP), and Lawrence and Arthur (HLA). The study shows that the HEA estimator is as robust as the HKP estimator. However, the HEA estimator posed certain computational problems and required more storage and computing time.     

Formulating robust regression estimation as an optimum allocation problem

The Mallows-type estimator, one of the most reasonable bounded influence estimators, often downweights leverage points regardless of the magnitude of the corresponding residual, and this could imply a loss of efficiency. In this article, we consider whether the efficiency of this bounded influence estimator could be improved by regarding both the robust x -distance and the residual size. We develop a new robust procedure based on the ideas of the Mallows-type estimator and the general robust recipe, where data been cleaned by pulling outliers towards their fitted values. Our basic idea is to formulate the robust estimation as an allocation problem, where the objective function is a Huber-type "loss" function, but the pulling resource is restricted. Using a mathematical programming technique, the pulling resource is optimally allocated to influential points <$>({x}_i, y_i)<$> with respect to residual size and given weights, <$>w({x}_i)<$>. Three previously published approaches are compared to our proposal via simulated experiments. In the case of contaminated data by regression outliers and "good" leverage points, the proposed robust estimator is a reasonable bounded influence estimator concerning both efficiency and norm of bias. In addition, the proposed approach offers the potential to establish constraints for the regression parameters and also may potentially provide insight regarding outlier detection.     

Robust linear discriminant analysis using S‐estimators

The authors consider a robust linear discriminant function based on high breakdown location and covariance matrix estimators. They derive influence functions for the estimators of the parameters of the discriminant function and for the associated classification error. The most B‐robust estimator is determined within the class of multivariate S‐estimators. This estimator, which minimizes the maximal influence that an outlier can have on the classification error, is also the most B‐robust location S‐estimator. A comparison of the most B‐robust estimator with the more familiar biweight S‐estimator is made.     

Bounded influence nonlinear signed‐rank regression

In this paper we consider weighted generalized‐signed‐rank estimators of nonlinear regression coefficients. The generalization allows us to include popular estimators such as the least squares and least absolute deviations estimators but by itself does not give bounded influence estimators. Adding weights results in estimators with bounded influence function. We establish conditions needed for the consistency and asymptotic normality of the proposed estimator and discuss how weight functions can be chosen to achieve bounded influence function of the estimator. Real life examples and Monte Carlo simulation experiments demonstrate the robustness and efficiency of the proposed estimator. An example shows that the weighted signed‐rank estimator can be useful to detect outliers in nonlinear regression. The Canadian Journal of Statistics 40: 172–189; 2012 © 2012 Statistical Society of Canada     

稳健统计以及几种统计量的稳健性比较分析

稳健统计作为统计学的一个较为活跃的研究领域,是对传统统计方法的完善和补充。其中最为简单和通俗易懂的是统计量的稳健性。本文对几种常用的统计量进行深入地剖析,进而揭示其稳健性的不足。同时给出几种稳健统计量,并与传统的统计量进行比较。通过比较来展现稳健统计量的优势及其应用价值。     

M-estimators for isotonic regression

In this paper we propose a family of robust estimates for isotonic regression: isotonic M-estimators. We show that their asymptotic distribution is, up to an scalar factor, the same as that of Brunk's classical isotonic estimator. We also derive the influence function and the breakdown point of these estimates. Finally we perform a Monte Carlo study that shows that the proposed family includes estimators that are simultaneously highly efficient under Gaussian errors and highly robust when the error distribution has heavy tails.     

Robust variance estimators for fixed-effect estimates with hierarchical-likelihood

In longitudinal studies, robust sandwich variance estimators are often used, and are especially useful when model assumptions are in doubt. However, the usual sandwich estimator does not allow for models with crossed random effects. The hierarchical likelihood extends the idea of the sandwich estimator to models not currently covered. By simulation studies, we show that the new sandwich estimator is robust against heteroscedastic errors and against misspecification of overdispersion in the y | v component.     

Local M-estimation for Conditional Variance in Heteroscedastic Regression Models

In this article, we develop a local M-estimation for the conditional variance in heteroscedastic regression models. The estimator is based on the local linear smoothing technique and the M-estimation technique, and it is shown to be not only asymptotically equivalent to the local linear estimator but also robust. The consistency and asymptotic normality of the local M-estimator for the conditional variance in heteroscedastic regression models are obtained under mild conditions. The simulation studies demonstrate that the proposed estimators perform well in robustness.