The Extent of Gross Errors Eliminated by Robust Multiple Linear Regressions

Robust estimation methods can effectively eliminate the influence of gross errors on parameter estimation. However, the extent of gross errors eliminated (EGEE) by robust estimation methods is far-reaching. This article presents a new approach to determine EGEE by robust estimation method. Taking multiple linear regressions (2–5) as examples, simulation experiments were conducted to compare the EGEE of 14 frequently used robust estimation methods. This article confirms several additional efficient robust estimation methods for dealing with multiple linear regressions, as well as the minimum number of observations needed to eliminate gross errors in certain ranges completely.     

Robust and diagnostic regression analyses

Graphical methods of diagnostic regression analysis are applied to three examples in which least squares and robust regression analyses give substantially different results. The diagnostic tools lead to the identification of data deficiencies and model inadequacies. The analyses serve as a reminder that robust regressions depend upon the linear model and upon the scale in whicli the response is analysed. The robust analysis may also be sensitive to gross errors in one or more explanatory variables     

Two-Stage Bounded-lnfluence Estimators for Simultaneous-Equations Models

This article presents a class of estimators for linear structural models that are robust to heavytailed disturbance distributions, gross errors in either the endogenous or exogenous variables, and certain other model failures. The class of estimators modifies ordinary two-stage least squares by replacing each least squares regression by a bounded-influence regression. Conditions under which the estimators are qualitatively robust, consistent, and asymptotically normal are established, and an empirical example is presented.     

Robust Linear Calibration

We regard the simple linear calibration problem where only the response y of the regression line y = β₀ + β₁ t is observed with errors. The experimental conditions t are observed without error. For the errors of the observations y we assume that there may be some gross errors providing outlying observations. This situation can be modeled by a conditionally contaminated regression model. In this model the classical calibration estimator based on the least squares estimator has an unbounded asymptotic bias. Therefore we introduce calibration estimators based on robust one-step-M-estimators which have a bounded asymptotic bias. For this class of estimators we discuss two problems: The optimal estimators and their corresponding optimal designs. We derive the locally optimal solutions and show that the maximin efficient designs for non-robust estimation and robust estimation coincide.     

Bayesian composite quantile regression for linear mixed-effects models

Longitudinal data are commonly modeled with the normal mixed-effects models. Most modeling methods are based on traditional mean regression, which results in non robust estimation when suffering extreme values or outliers. Median regression is also not a best choice to estimation especially for non normal errors. Compared to conventional modeling methods, composite quantile regression can provide robust estimation results even for non normal errors. In this paper, based on a so-called pseudo composite asymmetric Laplace distribution (PCALD), we develop a Bayesian treatment to composite quantile regression for mixed-effects models. Furthermore, with the location-scale mixture representation of the PCALD, we establish a Bayesian hierarchical model and achieve the posterior inference of all unknown parameters and latent variables using Markov Chain Monte Carlo (MCMC) method. Finally, this newly developed procedure is illustrated by some Monte Carlo simulations and a case analysis of HIV/AIDS clinical data set.     

Estimation in a change-point non linear quantile model

This paper considers a non linear quantile model with change-points. The quantile estimation method, which as a particular case includes median model, is more robust with respect to other traditional methods when model errors contain outliers. Under relatively weak assumptions, the convergence rate and asymptotic distribution of change-point and of regression parameter estimators are obtained. Numerical study by Monte Carlo simulations shows the performance of the proposed method for non linear model with change-points.     

WLAD-LASSO method for robust estimation and variable selection in partially linear models

This paper focuses on robust estimation and variable selection for partially linear models. We combine the weighted least absolute deviation (WLAD) regression with the adaptive least absolute shrinkage and selection operator (LASSO) to achieve simultaneous robust estimation and variable selection for partially linear models. Compared with the LAD-LASSO method, the WLAD-LASSO method will resist to the heavy-tailed errors and outliers in the parametric components. In addition, we estimate the unknown smooth function by a robust local linear regression. Under some regular conditions, the theoretical properties of the proposed estimators are established. We further examine finite-sample performance of the proposed procedure by simulation studies and a real data example.     

Data tilting for time series

Summary. We develop a general methodology for tilting time series data. Attention is focused on a large class of regression problems, where errors are expressed through autoregressive processes. The class has a range of important applications and in the context of our work may be used to illustrate the application of tilting methods to interval estimation in regression, robust statistical inference and estimation subject to constraints. The method can be viewed as 'empirical likelihood with nuisance parameters'.     

Robust methods for personal-income distribution models

Statistical problems in modelling personal-income distributions include estimation procedures, testing, and model choice. Typically, the parameters of a given model are estimated by classical procedures such as maximum-likelihood and least-squares estimators. Unfortunately, the classical methods are very sensitive to model deviations such as gross errors in the data, grouping effects, or model misspecifications. These deviations can ruin the values of the estimators and inequality measures and can produce false information about the distribution of the personal income in a country. In this paper we discuss the use of robust techniques for the estimation of income distributions. These methods behave like the classical procedures at the model but are less influenced by model deviations and can be applied to general estimation problems.     

Robust adaptive Lasso for variable selection

The adaptive least absolute shrinkage and selection operator (Lasso) and least absolute deviation (LAD)-Lasso are two attractive shrinkage methods for simultaneous variable selection and regression parameter estimation. While the adaptive Lasso is efficient for small magnitude errors, LAD-Lasso is robust against heavy-tailed errors and severe outliers. In this article, we consider a data-driven convex combination of these two modern procedures to produce a robust adaptive Lasso, which not only enjoys the oracle properties, but synthesizes the advantages of the adaptive Lasso and LAD-Lasso. It fully adapts to different error structures including the infinite variance case and automatically chooses the optimal weight to achieve both robustness and high efficiency. Extensive simulation studies demonstrate a good finite sample performance of the robust adaptive Lasso. Two data sets are analyzed to illustrate the practical use of the procedure.     

Classical and robust orthogonal regression between parts of compositional data

The different parts (variables) of a compositional data set cannot be considered independent from each other, since only the ratios between the parts constitute the relevant information to be analysed. Practically, this information can be included in a system of orthonormal coordinates. For the task of regression of one part on other parts, a specific choice of orthonormal coordinates is proposed which allows for an interpretation of the regression parameters in terms of the original parts. In this context, orthogonal regression is appropriate since all compositional parts – also the explanatory variables – are measured with errors. Besides classical (least-squares based) parameter estimation, also robust estimation based on robust principal component analysis is employed. Statistical inference for the regression parameters is obtained by bootstrap; in the robust version the fast and robust bootstrap procedure is used. The methodology is illustrated with a data set from macroeconomics.     

A robust coefficient of determination for regression

To assess the quality of the fit in a multiple linear regression, the coefficient of determination or R² is a very simple tool, yet the most used by practitioners. Indeed, it is reported in most statistical analyzes, and although it is not recommended as a final model selection tool, it provides an indication of the suitability of the chosen explanatory variables in predicting the response. In the classical setting, it is well known that the least-squares fit and coefficient of determination can be arbitrary and/or misleading in the presence of a single outlier. In many applied settings, the assumption of normality of the errors and the absence of outliers are difficult to establish. In these cases, robust procedures for estimation and inference in linear regression are available and provide a suitable alternative.     

Cocaine Dependence Treatment Data: Methods for Measurement Error Problems With Predictors Derived From Stationary Stochastic Processes

In a cocaine dependence treatment study, we use linear and nonlinear regression models to model posttreatment cocaine craving scores and first cocaine relapse time. A subset of the covariates are summary statistics derived from baseline daily cocaine use trajectories, such as baseline cocaine use frequency and average daily use amount. These summary statistics are subject to estimation error and can therefore cause biased estimators for the regression coefficients. Unlike classical measurement error problems, the error we encounter here is heteroscedastic with an unknown distribution, and there are no replicates for the error-prone variables or instrumental variables. We propose two robust methods to correct for the bias: a computationally efficient method-of-moments-based method for linear regression models and a subsampling extrapolation method that is generally applicable to both linear and nonlinear regression models. Simulations and an application to the cocaine dependence treatment data are used to illustrate the efficacy of the proposed methods. Asymptotic theory and variance estimation for the proposed subsampling extrapolation method and some additional simulation results are described in the online supplementary material.     

Penalized inverse probability weighted estimators for weighted rank regression with missing covariates

Abstract

In this article, we study the variable selection and estimation for linear regression models with missing covariates. The proposed estimation method is almost as efficient as the popular least-squares-based estimation method for normal random errors and empirically shown to be much more efficient and robust with respect to heavy tailed errors or outliers in the responses and covariates. To achieve sparsity, a variable selection procedure based on SCAD is proposed to conduct estimation and variable selection simultaneously. The procedure is shown to possess the oracle property. To deal with the covariates missing, we consider the inverse probability weighted estimators for the linear model when the selection probability is known or unknown. It is shown that the estimator by using estimated selection probability has a smaller asymptotic variance than that with true selection probability, thus is more efficient. Therefore, the important Horvitz-Thompson property is verified for penalized rank estimator with the covariates missing in the linear model. Some numerical examples are provided to demonstrate the performance of the estimators.     

Linear Regression With Nested Errors Using Probability‐Linked Data

Probabilistic matching of records is widely used to create linked data sets for use in health science, epidemiological, economic, demographic and sociological research. Clearly, this type of matching can lead to linkage errors, which in turn can lead to bias and increased variability when standard statistical estimation techniques are used with the linked data. In this paper we develop unbiased regression parameter estimates to be used when fitting a linear model with nested errors to probabilistically linked data. Since estimation of variance components is typically an important objective when fitting such a model, we also develop appropriate modifications to standard methods of variance components estimation in order to account for linkage error. In particular, we focus on three widely used methods of variance components estimation: analysis of variance, maximum likelihood and restricted maximum likelihood. Simulation results show that our estimators perform reasonably well when compared to standard estimation methods that ignore linkage errors.     

A Note on Asymmetry and Robustness in Linear Regression

We discuss the assumption of symmetry in robust linear regression. It is important to distinguish between the intercept term and the slope parameters. Ordinary robust regression requires no assumption of symmetry when interest lies in slope parameters; computer programs, confidence intervals, standard errors, and so forth do not change because the errors are asymmetric. The situation is radically different for bounded-influence estimators. With the exception of the Mallows class, these estimators are inconsistent for slope when the errors are asymmetric.     

Uncertain population forecasting

"Errors in population forecasts arise from errors in the jump-off population and errors in the predictions of future vital rates. The propagation of these errors through the linear (Leslie) growth model is studied, and prediction intervals for future population are developed. For U.S. national forecasts, the prediction intervals are compared with the U.S. Census Bureau's high-low intervals." In order to assess the accuracy of the predictions of vital rates, the authors "derive the predictions from a parametric statistical model and estimate the extent of model misspecification and errors in parameter estimates. Subjective, expert opinion, so important in real forecasting, is incorporated with the technique of mixed estimation. A robust regression model is used to assess the effects of model misspecification."     

Robust group-Lasso for functional regression model

In this article, we consider the problem of selecting functional variables using the L1 regularization in a functional linear regression model with a scalar response and functional predictors, in the presence of outliers. Since the LASSO is a special case of the penalized least-square regression with L1 penalty function, it suffers from the heavy-tailed errors and/or outliers in data. Recently, Least Absolute Deviation (LAD) and the LASSO methods have been combined (the LAD-LASSO regression method) to carry out robust parameter estimation and variable selection simultaneously for a multiple linear regression model. However, variable selection of the functional predictors based on LASSO fails since multiple parameters exist for a functional predictor. Therefore, group LASSO is used for selecting functional predictors since group LASSO selects grouped variables rather than individual variables. In this study, we propose a robust functional predictor selection method, the LAD-group LASSO, for a functional linear regression model with a scalar response and functional predictors. We illustrate the performance of the LAD-group LASSO on both simulated and real data.     

Penalized weighted composite quantile regression in the linear regression model with heavy-tailed autocorrelated errors

In this paper, a penalized weighted composite quantile regression estimation procedure is proposed to estimate unknown regression parameters and autoregression coefficients in the linear regression model with heavy-tailed autoregressive errors. Under some conditions, we show that the proposed estimator possesses the oracle properties. In addition, we introduce an iterative algorithm to achieve the proposed optimization problem, and use a data-driven method to choose the tuning parameters. Simulation studies demonstrate that the proposed new estimation method is robust and works much better than the least squares based method when there are outliers in the dataset or the autoregressive error distribution follows heavy-tailed distributions. Moreover, the proposed estimator works comparably to the least squares based estimator when there are no outliers and the error is normal. Finally, we apply the proposed methodology to analyze the electricity demand dataset.