A performance counterexample of Billor–Chatterjee–Hadi procedure and an improvement proposal for robust regression

The BCH procedure introduced by Billor, Chatterjee, and Hadi for fitting linear models was found to be inefficient for y-outliers in the presence of a high perturbation level. We propose to modify the first step of the BCH procedure, so that the robust distances are computed on the matrix Z = (y, X) of the basic subset. The performance of the present note procedure (PNP), as compared to the BCH procedure and the ordinary least-square (OLS) method, was studied by processing several datasets used in the literature for robust regression and by performing a Monte Carlo experiment. PNP performs better particularly with datasets having high perturbation.     

Descriptive tests for the identification of outliers in the errors in variables linear model

In this paper the most commonly used diagnostic criteria for the identification of outliers or leverage points in the ordinary regression model are reviewed. Their use in the context of the errors-in-variables (e.v.) linear model is discussed and evidence is given that under the e.v. model assumptions the distinction between outliers and leverage points no longer exists.     

Bounded-leverage weights for robust regression estimators

Both the least squares estimator and M-estimators of regression coefficients are susceptible to distortion when high leverage points occur among the predictor variables in a multiple linear regression model. In this article a weighting scheme which enables one to bound the leverage values of a weighted matrix of predictor variables is proposed. Bounded-leverage weighting of the predictor variables followed by M-estimation of the regression coefficients is shown to be effective in protecting against distortion due to extreme predictor-variable values, extreme response values, or outlier-induced multieollinearites. Bounded-leverage estimators can also protect against distortion by small groups of high leverage points.     

AN IMPROVED COMPOUND ESTIMATOR FOR ROBUST REGRESSION

ABSTRACT

Advances in statistical computing software have led to a substantial increase in the use of ordinary least squares (OLS) regression models in the engineering and applied statistics communities. Empirical evidence suggests that data sets can routinely have 10% or more outliers in many processes. Unfortunately, these outliers typically will render the OLS parameter estimates useless. The OLS diagnostic quantities and graphical plots can reliably identify a few outliers; however, they significantly lose power with increasing dimension and number of outliers. Although there have been recent advances in the methods that detect multiple outliers, improvements are needed in regression estimators that can fit well in the presence of outliers. We introduce a robust regression estimator that performs well regardless of outlier quantity and configuration. Our studies show that the best available estimators are vulnerable when the outliers are extreme in the regressor space (high leverage). Our proposed compound estimator modifies recently published methods with an improved initial estimate and measure of leverage. Extensive performance evaluations indicate that the proposed estimator performs the best and consistently fits the bulk of the data when outliers are present. The estimator, implemented in standard software, provides researchers and practitioners a tool for the model-building process to protect against the severe impact from multiple outliers.     

Computing least trimmed squares regression with the forward search

Least trimmed squares (LTS) provides a parametric family of high breakdown estimators in regression with better asymptotic properties than least median of squares (LMS) estimators. We adapt the forward search algorithm of Atkinson (1994) to LTS and provide methods for determining the amount of data to be trimmed. We examine the efficiency of different trimming proportions by simulation and demonstrate the increasing efficiency of parameter estimation as larger proportions of data are fitted using the LTS criterion. Some standard data examples are analysed. One shows that LTS provides more stable solutions than LMS.     

Geometric median and its application in the identification of multiple outliers

Geometric mean (GM) is having growing and wider applications in statistical data analysis as a measure of central tendency. It is generally believed that GM is less sensitive to outliers than the arithmetic mean (AM) but we suspect likewise the AM the GM may also suffer a huge set back in the presence of outliers, especially when multiple outliers occur in a data. So far as we know, not much work has been done on the robustness issue of GM. In quest of a simple robust measure of central tendency, we propose the geometric median (GMed) in this paper. We show that the classical GM has only 0% breakdown point while it is 50% for the proposed GMed. Numerical examples also support our claim that the proposed GMed is unaffected in the presence of multiple outliers and can maintain the highest possible 50% breakdown. Later we develop a new method for the identification of multiple outliers based on this proposed GMed. A variety of numerical examples show that the proposed method can successfully identify all potential outliers while the traditional GM fails to do so.     

A robust diagnostic plot for explanatory variables under model mis-specification

A typical added variable plot is a commonly used plot in assessing the accuracy of a normal linear model. This plot is often used to evaluate the effect of adding an explanatory variable into the model and to detect possibly high leverage points or influential observations on the added variable. However, this type of plot is generally in doubt, once the normal distributional assumptions are violated. In this article, we extend the robust likelihood technique introduced by Royall and Tsou [11] to propose a robust added variable plot. The validity of this diagnostic plot requires no knowledge of the true underlying distributions so long as their second moments exist. The usefulness of the robust graphical approach is demonstrated through a few illustrations and simulations.     

Regressor space outliers in ridge regression

A simple analytical expression is derived for leverage in ridge regression. Leverage is shown to be a monotonically decreasing function of the value of the ridge parameter. This reduction in leverage is greatest for those observations lying substantially in the direction of the minor principal axes. Thus, ridge estimation copes with outliers in regressor space by downweighting their influence. A brief illustration is provided.     

Robust automatic methods for outlier and error detection

Summary.  Editing in surveys of economic populations is often complicated by the fact that outliers due to errors in the data are mixed in with correct, but extreme, data values. We describe and evaluate two automatic techniques for the identification of errors in such long-tailed data distributions. The first is a forward search procedure based on finding a sequence of error-free subsets of the error-contaminated data and then using regression modelling within these subsets to identify errors. The second uses a robust regression tree modelling procedure to identify errors. Both approaches can be implemented on a univariate basis or on a multivariate basis. An application to a business survey data set that contains a mix of extreme errors and true outliers is described.     

Monotone support vector quantile regression

Quantile regression (QR) models have received a great deal of attention in both the theoretical and applied statistical literature. In this paper we propose support vector quantile regression (SVQR) with monotonicity restriction, which is easily obtained via the dual formulation of the optimization problem. We also provide the generalized approximate cross validation method for choosing the hyperparameters which affect the performance of the proposed SVQR. The experimental results for the synthetic and real data sets confirm the successful performance of the proposed model.     

Minicommunication computation of moments of generalized ridge estimators

This paper deals with a formal identification of outliers in regression based on tests of hypotheses. The hypothesis is not the standard one but is based on performance criteria that relates to the coefficient estimation and predictive capabilities of the model. The cri-teria include the trace of the mean square error matrix on the coefficients and integrated mean square error of prediction. Both the mean shift outlier model and the variance in-flation model are discussed.     

Approximate inference in heteroskedastic regressions: A numerical evaluation

The commonly made assumption that all stochastic error terms in the linear regression model share the same variance (homoskedasticity) is oftentimes violated in practical applications, especially when they are based on cross-sectional data. As a precaution, a number of practitioners choose to base inference on the parameters that index the model on tests whose statistics employ asymptotically correct standard errors, i.e. standard errors that are asymptotically valid whether or not the errors are homoskedastic. In this paper, we use numerical integration methods to evaluate the finite-sample performance of tests based on different (alternative) heteroskedasticity-consistent standard errors. Emphasis is placed on a few recently proposed heteroskedasticity-consistent covariance matrix estimators. Overall, the results favor the HC4 and HC5 heteroskedasticity-robust standard errors. We also consider the use of restricted residuals when constructing asymptotically valid standard errors. Our results show that the only test that clearly benefits from such a strategy is the HC0 test.