M-estimators for isotonic regression

In this paper we propose a family of robust estimates for isotonic regression: isotonic M-estimators. We show that their asymptotic distribution is, up to an scalar factor, the same as that of Brunk's classical isotonic estimator. We also derive the influence function and the breakdown point of these estimates. Finally we perform a Monte Carlo study that shows that the proposed family includes estimators that are simultaneously highly efficient under Gaussian errors and highly robust when the error distribution has heavy tails.     

A new modified Jackknifed estimator for the Poisson regression model

The Poisson regression is very popular in applied researches when analyzing the count data. However, multicollinearity problem arises for the Poisson regression model when the independent variables are highly intercorrelated. Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators and some methods for estimating the ridge parameter k in the Poisson regression have been proposed. It has been found that some estimators are better than the commonly used maximum-likelihood (ML) estimator and some other RR estimators. In this study, the modified Jackknifed Poisson ridge regression (MJPR) estimator is proposed to remedy the multicollinearity. A simulation study and a real data example are provided to evaluate the performance of estimators. Both mean-squared error and the percentage relative error are considered as the performance criteria. The simulation study and the real data example results show that the proposed MJPR method outperforms the Poisson ridge regression, Jackknifed Poisson ridge regression and the ML in all of the different situations evaluated in this paper.     

The empirical Bayes estimators of the parameter of the Poisson distribution with a conjugate gamma prior under Stein's loss function

For the hierarchical Poisson and gamma model, we calculate the Bayes posterior estimator of the parameter of the Poisson distribution under Stein's loss function which penalizes gross overestimation and gross underestimation equally and the corresponding Posterior Expected Stein's Loss (PESL). We also obtain the Bayes posterior estimator of the parameter under the squared error loss and the corresponding PESL. Moreover, we obtain the empirical Bayes estimators of the parameter of the Poisson distribution with a conjugate gamma prior by two methods. In numerical simulations, we have illustrated: The two inequalities of the Bayes posterior estimators and the PESLs; the moment estimators and the Maximum Likelihood Estimators (MLEs) are consistent estimators of the hyperparameters; the goodness-of-fit of the model to the simulated data. The numerical results indicate that the MLEs are better than the moment estimators when estimating the hyperparameters. Finally, we exploit the attendance data on 314 high school juniors from two urban high schools to illustrate our theoretical studies.     

A new method of imputation in survey sampling

In this paper, an alternative estimator of population mean in the presence of non-response has been suggested which comes in the form of Walsh's estimator. The estimator of mean obtained from the proposed technique remains better than the estimators obtained from ratio or mean methods of imputation. The mean-squared error (MSE) of the resultant estimator is less than that of the estimator obtained on the basis of ratio method of imputation for the optimum choice of parameters. An estimator for estimating a parameter involved in the process of new method of imputation has been discussed. A suggestion to form ‘warm deck’ method of imputation has been suggested. The MSE expressions for the proposed estimators have been derived analytically and compared empirically. The work has been extended to the case of multi-auxiliary information to be used for imputation. Numerical illustrations are also provided.     

A comparison of some robust,adaptive, and partially adaptive estimators of regression models

Numerous estimation techniques for regression models have been proposed. These procedures differ in how sample information is used in the estimation procedure. The efficiency of least squares (OLS) estimators implicity assumes normally distributed residuals and is very sensitive to departures from normality, particularly to "outliers" and thick-tailed distributions. Lead absolute deviation (LAD) estimators are less sensitive to outliers and are optimal for laplace random disturbances, but not for normal errors. This paper reports monte carlo comparisons of OLS,LAD, two robust estimators discussed by huber, three partially adaptiveestimators, newey's generalized method of moments estimator, and an adaptive maximum likelihood estimator based on a normal kernal studied by manski. This paper is the first to compare the relative performance of some adaptive robust estimators (partially adaptive and adaptive procedures) with some common nonadaptive robust estimators. The partially adaptive estimators are based on three flxible parametric distributions for the errors. These include the power exponential (Box-Tiao) and generalized t distributions, as well as a distribution for the errors, which is not necessarily symmetric. The adaptive procedures are "fully iterative" rather than one step estimators. The adaptive estimators have desirable large sample properties, but these properties do not necessarily carry over to the small sample case.

The monte carlo comparisons of the alternative estimators are based on four different specifications for the error distribution: a normal, a mixture of normals (or variance-contaminated normal), a bimodal mixture of normals, and a lognormal. Five hundred samples of 50 are used. The adaptive and partially adaptive estimators perform very well relative to the other estimation procedures considered, and preliminary results suggest that in some important cases they can perform much better than OLS with 50 to 80% reductions in standard errors.

     

Empirical distribution function under heteroscedasticity

Neglecting heteroscedasticity of error terms may imply the wrong identification of a regression model (see appendix). Employment of (heteroscedasticity resistent) White's estimator of covariance matrix of estimates of regression coefficients may lead to the correct decision about the significance of individual explanatory variables under heteroscedasticity. However, White's estimator of covariance matrix was established for least squares (LS)-regression analysis (in the case when error terms are normally distributed, LS- and maximum likelihood (ML)-analysis coincide and hence then White's estimate of covariance matrix is available for ML-regression analysis, tool). To establish White's-type estimate for another estimator of regression coefficients requires Bahadur representation of the estimator in question, under heteroscedasticity of error terms. The derivation of Bahadur representation for other (robust) estimators requires some tools. As the key too proved to be a tight approximation of the empirical distribution function (d.f.) of residuals by the theoretical d.f. of the error terms of the regression model. We need the approximation to be uniform in the argument of d.f. as well as in regression coefficients. The present paper offers this approximation for the situation when the error terms are heteroscedastic.     

Modified maximum likelihood estimator under the Jones and Faddy's skew t-error distribution for censored regression model

It is well-known that classical Tobit estimator of the parameters of the censored regression (CR) model is inefficient in case of non-normal error terms. In this paper, we propose to use the modified maximum likelihood (MML) estimator under the Jones and Faddy''s skew t-error distribution, which covers a wide range of skew and symmetric distributions, for the CR model. The MML estimators, providing an alternative to the Tobit estimator, are explicitly expressed and they are asymptotically equivalent to the maximum likelihood estimator. A simulation study is conducted to compare the efficiencies of the MML estimators with the classical estimators such as the ordinary least squares, Tobit, censored least absolute deviations and symmetrically trimmed least squares estimators. The results of the simulation study show that the MML estimators work well among the others with respect to the root mean square error criterion for the CR model. A real life example is also provided to show the suitability of the MML methodology.     

Robust estimation of the SUR model

This paper proposes robust regression to solve the problem of outliers in seemingly unrelated regression (SUR) models. The authors present an adaptation of S‐estimators to SUR models. S‐estimators are robust, have a high breakdown point and are much more efficient than other robust regression estimators commonly used in practice. Furthermore, modifications to Ruppert's algorithm allow a fast evaluation of them in this context. The classical example of U.S. corporations is revisited, and it appears that the procedure gives an interesting insight into the problem.     

M-estimators of some GARCH-type models; computation and application

In this paper, we consider robust M-estimation of time series models with both symmetric and asymmetric forms of heteroscedasticity related to the GARCH and GJR models. The class of estimators includes least absolute deviation (LAD), Huber’s, Cauchy and B-estimator as well as the well-known quasi maximum likelihood estimator (QMLE). Extensive simulations are used to check the relative performance of these estimators in both models and the weighted resampling methods are used to approximate the sampling distribution of M-estimators. Our study indicates that there are estimators that can perform better than QMLE and even outperform robust estimator such as LAD when the error distribution is heavy-tailed. These estimators are also applied to real data sets.     

Some maximum-indifference estimators for the slope of a univariate linear model

As known, the least-squares estimator of the slope of a univariate linear model sets to zero the covariance between the regression residuals and the values of the explanatory variable. To prevent the estimation process from being influenced by outliers, which can be theoretically modelled by a heavy-tailed distribution for the error term, one can substitute covariance with some robust measures of association, for example Kendall's tau in the popular Theil–Sen estimator. In a scarcely known Italian paper, Cifarelli [(1978), ‘La Stima del Coefficiente di Regressione Mediante l'Indice di Cograduazione di Gini’, Rivista di matematica per le scienze economiche e sociali, 1, 7–38. A translation into English is available at http://arxiv.org/abs/1411.4809 and will appear in Decisions in Economics and Finance] shows that a gain of efficiency can be obtained by using Gini's cograduation index instead of Kendall's tau. This paper introduces a new estimator, derived from another association measure recently proposed. Such a measure is strongly related to Gini's cograduation index, as they are both built to vanish in the general framework of indifference. The newly proposed estimator is shown to be unbiased and asymptotically normally distributed. Moreover, all considered estimators are compared via their asymptotic relative efficiency and a small simulation study. Finally, some indications about the performance of the considered estimators in the presence of contaminated normal data are provided.     

A comparison of robust alternatives to Hotelling’s T 2 control chart

Control charts are one of the widest used techniques in statistical process control. In Phase I, historical observations are analysed in order to construct a control chart. Because of the existence of multiple outliers that are undetected by control charts such as Hotelling’s T ² due to the masking effect, robust alternatives to Hotelling’s T ² have been developed based on minimum volume ellipsoid (MVE) estimators, minimum covariance determinant (MCD) estimators, reweighted MCD estimators or trimmed estimators. In this paper, we use a simulation study to analyse the performance of each alternative in various situations and offer guidance for the correct use of each estimator.     

The exact risks of some pre-test and stein-type regression estimators umder balanced loss

In regression analysis we are often interested in using an estimator which is “precise” and which simultaneously provides a model with “good fit”, In this paper we consider the risk properties of several estimators of the regression coefficient vector "trader “balanced” loss, This loss function (Zellner, 1994) reflects both of the described attributes. Under a particular form of balanced loss, we derive the predictive risk of the pre-test estimator which results after a test for exact linear restrictions on the coefficient vector. The corresponding risks of Stein-rule and positive-part Stein-rale estimators are also established. The risks based on loss functions which allow only for estimation precision, or only for goodness of fit, are special cases of our results, and we draw appropriate comparisons, In particular, we show that some of the well-known results under (quadratic) precision-only loss are not robust to our generalization of the loss function     

Estimators in capture–recapture studies with two sources

This paper investigates the applications of capture–recapture methods to human populations. Capture–recapture methods are commonly used in estimating the size of wildlife populations but can also be used in epidemiology and social sciences, for estimating prevalence of a particular disease or the size of the homeless population in a certain area. Here we focus on estimating the prevalence of infectious diseases. Several estimators of population size are considered: the Lincoln–Petersen estimator and its modified version, the Chapman estimator, Chao’s lower bound estimator, the Zelterman’s estimator, McKendrick’s moment estimator and the maximum likelihood estimator. In order to evaluate these estimators, they are applied to real, three-source, capture-recapture data. By conditioning on each of the sources of three source data, we have been able to compare the estimators with the true value that they are estimating. The Chapman and Chao estimators were compared in terms of their relative bias. A variance formula derived through conditioning is suggested for Chao’s estimator, and normal 95% confidence intervals are calculated for this and the Chapman estimator. We then compare the coverage of the respective confidence intervals. Furthermore, a simulation study is included to compare Chao’s and Chapman’s estimator. Results indicate that Chao’s estimator is less biased than Chapman’s estimator unless both sources are independent. Chao’s estimator has also the smaller mean squared error. Finally, the implications and limitations of the above methods are discussed, with suggestions for further development. We are grateful to the Medical Research Council for supporting this work.     

A simulation study of five biased estimators for straight line regression

Five biased estimators of the slope in straight line regression are considered. For each, the estimate of the “bias parameter”, k, is a function of N, the number of observations, and [rcirc]² , the square of the least squares estimate of the standardized slope, β. The estimators include that of Farebrother, the ridge estimator of Hoerl, Kennard, and Baldwin, Vinod's shrunken estimators., and a new modification of one of the latter. Properties of the estimators are studied for 13 combinations of N and 3. Results of simulation experiments provide empirical evidence concerning the values of means and variances of the biased estimators of the slope and estimates of the “bias parameter”, the mean square errors of the estimators, and the frequency of improvement relative to least squares. Adjustments to degrees of freedom in the biased regression analysis of variance table are also considered. An extension of the new modification to the case of p> 1 independent variables is presented in an Appendix.     

Robust linear discriminant analysis using S‐estimators

The authors consider a robust linear discriminant function based on high breakdown location and covariance matrix estimators. They derive influence functions for the estimators of the parameters of the discriminant function and for the associated classification error. The most B‐robust estimator is determined within the class of multivariate S‐estimators. This estimator, which minimizes the maximal influence that an outlier can have on the classification error, is also the most B‐robust location S‐estimator. A comparison of the most B‐robust estimator with the more familiar biweight S‐estimator is made.     

The Gauss—Markov Theorem and Random Regressors

In the standard linear regression model with independent, homoscedastic errors, the Gauss—Markov theorem asserts that = (X'X)^-1(X'y) is the best linear unbiased estimator of β and, furthermore, that is the best linear unbiased estimator of c'β for all p × 1 vectors c. In the corresponding random regressor model, X is a random sample of size n from a p-variate distribution. If attention is restricted to linear estimators of c'β that are conditionally unbiased, given X, the Gauss—Markov theorem applies. If, however, the estimator is required only to be unconditionally unbiased, the Gauss—Markov theorem may or may not hold, depending on what is known about the distribution of X. The results generalize to the case in which X is a random sample without replacement from a finite population.     

Developing a restricted two-parameter Liu-type estimator: A comparison of restricted estimators in the binary logistic regression model

In the context of estimating regression coefficients of an ill-conditioned binary logistic regression model, we develop a new biased estimator having two parameters for estimating the regression vector parameter β when it is subjected to lie in the linear subspace restriction Hβ = h. The matrix mean squared error and mean squared error (MSE) functions of these newly defined estimators are derived. Moreover, a method to choose the two parameters is proposed. Then, the performance of the proposed estimator is compared to that of the restricted maximum likelihood estimator and some other existing estimators in the sense of MSE via a Monte Carlo simulation study. According to the simulation results, the performance of the estimators depends on the sample size, number of explanatory variables, and degree of correlation. The superiority region of our proposed estimator is identified based on the biasing parameters, numerically. It is concluded that the new estimator is superior to the others in most of the situations considered and it is recommended to the researchers.     

A robust and efficient estimation method for nonparametric models with jump points

Nonparametric models with jump points have been considered by many researchers. However, most existing methods based on least squares or likelihood are sensitive when there are outliers or the error distribution is heavy tailed. In this article, a local piecewise-modal method is proposed to estimate the regression function with jump points in nonparametric models, and a piecewise-modal EM algorithm is introduced to estimate the proposed estimator. Under some regular conditions, the large-sample theory is established for the proposed estimators. Several simulations are presented to evaluate the performances of the proposed method, which shows that the proposed estimator is more efficient than the local piecewise-polynomial regression estimator in the presence of outliers or heavy tail error distribution. What is more, the proposed procedure is asymptotically equivalent to the local piecewise-polynomial regression estimator under the assumption that the error distribution is a Gaussian distribution. The proposed method is further illustrated via the sea-level pressures.     

A new estimator for the unit root

We compare the ordinary least squares, weighted symmetric, modified weighted symmetric (MWS), maximum likelihood, and our new modification for least squares (MLS) estimator for first-order autoregressive in the case of unit root using Monte Carlo method. The Monte Carlo study sheds some light on how well the estimators and the predictors perform on different samples sizes. We found that MLS estimator is less biased and has less mean squared error (MSE) than any other estimators, and MWS predictor error performs well, in the sense of MSE, than any other predictors’ methods. The sample percentiles for the distribution of the τ statistic for the first, second, and third periods in the future, for alternative estimators, are reported to know if it agrees with those of normal distribution or not.     

Robust Estimation of Parameters in Simple Linear Profiles Using M-Estimators

In many situations, the quality of a process or product may be better characterized and summarized by a relationship between the response variable and one or more explanatory variables. Parameter estimation is the first step in constructing control charts. Outliers may hamper proper classical estimators and lead to incorrect conclusions. To remedy the problem of outliers, robust methods have been developed recently. In this article, a robust method is introduced for estimating the parameters of simple linear profiles. Two weight functions, Huber and Bisquare, are applied in the estimation algorithm. In addition, a method for robust estimation of the error terms variance is proposed. Simulation studies are done to investigate and evaluate the performance of the proposed estimator, as well as the classical one, in the presence and absence of outliers under different scenarios by the means of MSE criterion. The results reveal that the robust estimators proposed in this research perform as well as classical estimators in the absence of outliers and even considerably better when outliers exist. The maximum value of variance estimate in one scenario obtained from classical estimator is 10.9, while this value is 1.66 and 1.27 from proposed robust estimators when its actual value is 1.