A comparison of some robust,adaptive, and partially adaptive estimators of regression models

Numerous estimation techniques for regression models have been proposed. These procedures differ in how sample information is used in the estimation procedure. The efficiency of least squares (OLS) estimators implicity assumes normally distributed residuals and is very sensitive to departures from normality, particularly to "outliers" and thick-tailed distributions. Lead absolute deviation (LAD) estimators are less sensitive to outliers and are optimal for laplace random disturbances, but not for normal errors. This paper reports monte carlo comparisons of OLS,LAD, two robust estimators discussed by huber, three partially adaptiveestimators, newey's generalized method of moments estimator, and an adaptive maximum likelihood estimator based on a normal kernal studied by manski. This paper is the first to compare the relative performance of some adaptive robust estimators (partially adaptive and adaptive procedures) with some common nonadaptive robust estimators. The partially adaptive estimators are based on three flxible parametric distributions for the errors. These include the power exponential (Box-Tiao) and generalized t distributions, as well as a distribution for the errors, which is not necessarily symmetric. The adaptive procedures are "fully iterative" rather than one step estimators. The adaptive estimators have desirable large sample properties, but these properties do not necessarily carry over to the small sample case.

The monte carlo comparisons of the alternative estimators are based on four different specifications for the error distribution: a normal, a mixture of normals (or variance-contaminated normal), a bimodal mixture of normals, and a lognormal. Five hundred samples of 50 are used. The adaptive and partially adaptive estimators perform very well relative to the other estimation procedures considered, and preliminary results suggest that in some important cases they can perform much better than OLS with 50 to 80% reductions in standard errors.

     

A comparison of some robust, adaptive, and partially adaptive estimators of regression models

Numerous estimation techniques for regression models have been proposed. These procedures differ in how sample information is used in the estimation procedure. The efficiency of least squares (OLS) estimators implicity assumes normally distributed residuals and is very sensitive to departures from normality, particularly to "outliers" and thick-tailed distributions. Lead absolute deviation (LAD) estimators are less sensitive to outliers and are optimal for laplace random disturbances, but not for normal errors. This paper reports monte carlo comparisons of OLS,LAD, two robust estimators discussed by huber, three partially adaptiveestimators, newey's generalized method of moments estimator, and an adaptive maximum likelihood estimator based on a normal kernal studied by manski. This paper is the first to compare the relative performance of some adaptive robust estimators (partially adaptive and adaptive procedures) with some common nonadaptive robust estimators. The partially adaptive estimators are based on three flxible parametric distributions for the errors. These include the power exponential (Box-Tiao) and generalized t distributions, as well as a distribution for the errors, which is not necessarily symmetric. The adaptive procedures are "fully iterative" rather than one step estimators. The adaptive estimators have desirable large sample properties, but these properties do not necessarily carry over to the small sample case.

The monte carlo comparisons of the alternative estimators are based on four different specifications for the error distribution: a normal, a mixture of normals (or variance-contaminated normal), a bimodal mixture of normals, and a lognormal. Five hundred samples of 50 are used. The adaptive and partially adaptive estimators perform very well relative to the other estimation procedures considered, and preliminary results suggest that in some important cases they can perform much better than OLS with 50 to 80% reductions in standard errors.     

Asymptotic efficiencies of estiamtors of location a comutational study

Asymptotic efficiencies of four classes of estimators of location are evaluated for a family of distributions consisting of t, lambda and contaminated normal densities. For the class of estimators derived from signed rank tests, maximin efficiencies between pairs of distributions in the family are computed using a formula of Gastwirth ( 1966 ). Asymptotic efficiencies are also evaluated for the scale dependent estimators of the form proposed by Hubcr ( 1964 ) and the efficiencies of procedures utilizing interquantiie ranges.are evaluated. Efficiencies of linear estimators such as trimmed means, BLUE's for the lambda family are computed for each density considered. Efficiencies of linear, polynomial and trigonometric approximations to BLUE weight functions are determined. Using the method of Birnbaum and Laska ( 1967 ) maximin efficiencies are computed using four linear or polynomial terms. On the basis of comparisons of these numerical results, suggestions for robust estimators are given     

Tukey’s M-estimator of the Poisson parameter with a special focus on small means

We treat robust M-estimators for independent and identically distributed Poisson data. We introduce modified Tukey M-estimators with bias correction and compare them to M-estimators based on the Huber function as well as to weighted likelihood and other estimators by simulation in case of clean data and data with outliers. In particular, we investigate the problem of combining robustness and high efficiencies at small Poisson means caused by the strong asymmetry of such Poisson distributions and propose a further estimator based on adaptive trimming. The advantages of the constructed estimators are illustrated by an application to smoothing count data with a time varying mean and level shifts.     

Robust pairwise multiple comparisons under short-tailed symmetric distributions

In one-way ANOVA, most of the pairwise multiple comparison procedures depend on normality assumption of errors. In practice, errors have non-normal distributions so frequently. Therefore, it is very important to develop robust estimators of location and the associated variance under non-normality. In this paper, we consider the estimation of one-way ANOVA model parameters to make pairwise multiple comparisons under short-tailed symmetric (STS) distribution. The classical least squares method is neither efficient nor robust and maximum likelihood estimation technique is problematic in this situation. Modified maximum likelihood (MML) estimation technique gives the opportunity to estimate model parameters in closed forms under non-normal distributions. Hence, the use of MML estimators in the test statistic is proposed for pairwise multiple comparisons under STS distribution. The efficiency and power comparisons of the test statistic based on sample mean, trimmed mean, wave and MML estimators are given and the robustness of the test obtained using these estimators under plausible alternatives and inlier model are examined. It is demonstrated that the test statistic based on MML estimators is efficient and robust and the corresponding test is more powerful and having smallest Type I error.     

Generalised Rank Regression Estimator with Standard Error Adjusted Lasso

One of the standard variable selection procedures in multiple linear regression is to use a penalisation technique in least‐squares (LS) analysis. In this setting, many different types of penalties have been introduced to achieve variable selection. It is well known that LS analysis is sensitive to outliers, and consequently outliers can present serious problems for the classical variable selection procedures. Since rank‐based procedures have desirable robustness properties compared to LS procedures, we propose a rank‐based adaptive lasso‐type penalised regression estimator and a corresponding variable selection procedure for linear regression models. The proposed estimator and variable selection procedure are robust against outliers in both response and predictor space. Furthermore, since rank regression can yield unstable estimators in the presence of multicollinearity, in order to provide inference that is robust against multicollinearity, we adjust the penalty term in the adaptive lasso function by incorporating the standard errors of the rank estimator. The theoretical properties of the proposed procedures are established and their performances are investigated by means of simulations. Finally, the estimator and variable selection procedure are applied to the Plasma Beta‐Carotene Level data set.     

Partially Adaptive Estimation of the Censored Regression Model

Data censoring causes ordinary least squares estimates of linear models to be biased and inconsistent. Tobit, semiparametric, and partially adaptive estimators have been considered as possible solutions. This paper proposes several new partially adaptive estimators that cover a wide range of distributional characteristics. A simulation study is used to investigate the estimators’ relative efficiency in these settings. The partially adaptive censored regression estimators have little efficiency loss for censored normal errors and may outperform Tobit and semiparametric estimators considered for non-normal distributions. An empirical example of out-of-pocket expenditures for a health insurance plan provides an example, which supports these results.     

Marshall–Olkin distribution: parameter estimation and application to cancer data

In this study, as alternatives to the maximum likelihood (ML) and the frequency estimators, we propose robust estimators for the parameters of Zipf and Marshall–Olkin Zipf distributions. A small simulation study is given to illustrate the performance of the proposed estimators. We apply the proposed estimators to a real data set from cancer research to illustrate the performance of the proposed estimators over the ML, moments and frequency estimators. We observe that the robust estimators have superiority over the frequency estimators based on classical sample mean.     

RANKED SET SAMPLING FROM LOCATION-SCALE FAMILIES OF SYMMETRIC DISTRIBUTIONS

Statistical inference based on ranked set sampling has primarily been motivated by nonparametric problems. However, the sampling procedure can provide an improved estimator of the population mean when the population is partially known. In this article, we consider estimation of the population mean and variance for the location-scale families of distributions. We derive and compare different unbiased estimators of these parameters based on rindependent replications of a ranked set sample of size n.Large sample properties, along with asymptotic relative efficiencies, help identify which estimators are best suited for different location-scale distributions.     

Studentized robust statistics for,main effects in a two-factor manova

Multiresponse experiments in two-faoior manova are considered. StalibLical procedures of the test and estimation, based on studentized robust statistics. for location parameters in the models arc piupused. Large sample properties of their procedures as the cell sizes tend to infinity are investigated. Although Fisher's consistency is assumed in the theory ol ili-estimators, it is not needed. in this paper. For the univariate case, it is found that the asymptotic relative efficiencies (ARE's) of the proposed procedures relative to classical procedures agrees with the classical A/Sisresults of Huber's one sample Mestimator relative to the sample mean. By simulation studies, it can be seen that the proposed estimators are more efficient than the least squares estimators except for the case where the underlying distribution is normal     

Some robust anova procedures under heteroscedasticity and nonnormality

In this paper we consider the problem of comparing several means under heteroscedasticity and nonnormality. By combining Huber‘s M-estimators with the Brown-Forsythe test, several robust procedures were developed; these procedures were compared through computer simulation studies with the Tan-Tabatabai procedure which was developed by combining Tiku's MML estimators with the Brown-Forsythe test. The numerical results indicate clearly that the Tan-Tabatabai procedure is considerably more powerful than tests based on Huber's M-estimators over a wide range of nonnormal distributions.     

Bimatrix variate gamma-beta distributions

Two bimatrix distributions with beta and gamma marginals are introduced. Various properties (including product moments of determinants and traces, entropies, marginal distributions) are derived. Parameter estimation by the method of maximum likelihood is discussed. The performance and efficiencies of the maximum likelihood estimators and the associated confidence intervals are assessed by simulation. The efficiencies are compared versus those for the maximum likelihood estimators and the associated confidence intervals based on matrix variate gamma distributions. A discussion of possible applications of the bimatrix distributions is given.     

Robust test for means when population variances are unequal

We consider the problem of testing the equality of two population means when the population variances are not necessarily equal. We propose a Welch-type statistic, say T^* _c, based on Tiku!s ‘1967, 1980’ modified maximum likelihood estimators, and show that this statistic is robust to symmetric and moderately skew distributions. We investigate the power properties of the statistic T^* _c; T^* _c clearly seems to be more powerful than Yuen's ‘1974’ Welch-type robust statistic based on the trimmed sample means and the matching sample variances. We show that the analogous statistics based on the ‘adaptive’ robust estimators give misleading Type I errors. We generalize the results to testing linear contrasts among k population means     

Rare and clustered population estimation using the adaptive cluster sampling with some robust measures

The use of robust measures helps to increase the precision of the estimators, especially for the estimation of extremely skewed distributions. In this article, a generalized ratio estimator is proposed by using some robust measures with single auxiliary variable under the adaptive cluster sampling (ACS) design. We have incorporated tri-mean (TM), mid-range (MR) and Hodges-Lehman (HL) of the auxiliary variable as robust measures together with some conventional measures. The expressions of bias and mean square error (MSE) of the proposed generalized ratio estimator are derived. Two types of numerical study have been conducted using artificial clustered population and real data application to examine the performance of the proposed estimator over the usual mean per unit estimator under simple random sampling (SRS). Related results of the simulation study show that the proposed estimators provide better estimation results on both real and artificial population over the competing estimators.     

Robustness estimators of location:a classification of linear and adaptive estimators

Real-world data sets may be described in terms similar to trauma cases- 'messy' with 'high morbidity'. Alternative estimators to the traditional mean are examined via a simulation study over a wide range of both symmetric and asymmetric distributions. These alternative estimators are data depenmdent and, in most cases, represent data far better than the usual mean. Princeton and post-Princeton linear and adaptive estimators of location are summarized, and a classification scheme based on an ancillary or selector statistic is proposed. The computational formulae for the collection of estimators have been standardized, as have the ancillary statistics. We classify these estimators by their computational form, give the computational formulae for each in a standardized notation, evaluate the subclass of estimators, and identify our 'winner' in that class.     

Modified regression estimators using robust regression methods and covariance matrices in stratified random sampling

Abstract

This article proposes new regression-type estimators by considering Tukey-M, Hampel M, Huber MM, LTS, LMS and LAD robust methods and MCD and MVE robust covariance matrices in stratified sampling. Theoretically, we obtain the mean square error (MSE) for these estimators. We compare the efficiencies based on MSE equations, between the proposed estimators and the traditional combined and separate regression estimators. As a result of these comparisons, we observed that our proposed estimators give more efficient results than traditional approaches. And, these theoretical results are supported with the aid of numerical examples and simulation based on data sets that include outliers.     

Effect of a new selection procedure on adaptive estimation

Adaptive estimation of parameters of some failure time distributionsis considered. A new procedure named the F-procedure has beendeveloped for selecting an appropriate model out of two possible models by Pandey et.al. (1991). Applying this F-procedure adaptive estimatorsof parameters of exponential, Wei bull, inverse Gaussian (IG) and Wald failure time distributions have been proposed in this paper. Comparison of these estimators has been undertaken with MLE's of the respective parameters and with some previous adaptiveestimators by simulation of samples using the Monte Carlo method.Adaptive estimation of parameters of some failure time distributions is considered. A new procedure named the F-procedure has been developedfor selecting an appropriate model out of two possible models by Pandey et.al. (1991). Applying this F-procedure adaptive estimators of parameters of exponential, Wei bull, inverse Gaussian (IG) and Wald failure time distributions have been proposed in this paper. Comparison of these estimators has been undertaken with MLE's of the respective parameters and with some previous adaptive estimators by simulation of samples using the Monte Carlo method.     

Ranked set sampling with unequal samples for skew distributions

A ranked set sampling procedure with unequal samples for positively skew distributions (RSSUS) is proposed and used to estimate the population mean. The estimators based on RSSUS are compared with the estimators based on ranked set sampling (RSS) and median ranked set sampling (MRSS) procedures. It is observed that the relative precisions of the estimators based on RSSUS are higher than those of the estimators based on RSS and MRSS procedures.     

Tail-Adaptive Location Rank Test for the Generalized Secant Hyperbolic Distribution

The generalized secant hyperbolic distribution (GSHD) was recently introduced as a modeling tool in data analysis. The GSHD is a unimodal distribution that is completely specified by location, scale, and shape parameters. It has also been shown elsewhere that the rank procedures of location are regular, robust, and asymptotically fully efficient. In this article, we study certain tail weight measures for the GSHD and introduce a tail-adaptive rank procedure of location based on those tail weight measures. We investigate the properties of the new adaptive rank procedure and compare it to some conventional estimators.     

An Extension of the Normal/Independent Family for Studies of Robust Estimators

The family of normal scale mixture distributions, also called the Normal/Independent family, has been used for efficient Monte Carlo studies of robust estimators. The distributions in this family are unimodal. The Normal/Independent family is extended by introducing a location mixing in addition to the scale mixing. Distributions in this extension may be nonunimodal. The asymptotic variances of robust estimators of location are compared using the distributions from the extension. A Monte Carlo swindle similar to the one used in the Princeton study is given for the extended family. A small simulation study demonstrates the efficiency of the swindle. The swindle is compared with other swindle methods based on Fisher's score function and regression.