Robust ridge and robust Liu estimator for regression based on the LTS estimator

In the multiple linear regression analysis, the ridge regression estimator and the Liu estimator are often used to address multicollinearity. Besides multicollinearity, outliers are also a problem in the multiple linear regression analysis. We propose new biased estimators based on the least trimmed squares (LTS) ridge estimator and the LTS Liu estimator in the case of the presence of both outliers and multicollinearity. For this purpose, a simulation study is conducted in order to see the difference between the robust ridge estimator and the robust Liu estimator in terms of their effectiveness; the mean square error. In our simulations, the behavior of the new biased estimators is examined for types of outliers: X-space outlier, Y-space outlier, and X-and Y-space outlier. The results for a number of different illustrative cases are presented. This paper also provides the results for the robust ridge regression and robust Liu estimators based on a real-life data set combining the problem of multicollinearity and outliers.     

A new modified Jackknifed estimator for the Poisson regression model

The Poisson regression is very popular in applied researches when analyzing the count data. However, multicollinearity problem arises for the Poisson regression model when the independent variables are highly intercorrelated. Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators and some methods for estimating the ridge parameter k in the Poisson regression have been proposed. It has been found that some estimators are better than the commonly used maximum-likelihood (ML) estimator and some other RR estimators. In this study, the modified Jackknifed Poisson ridge regression (MJPR) estimator is proposed to remedy the multicollinearity. A simulation study and a real data example are provided to evaluate the performance of estimators. Both mean-squared error and the percentage relative error are considered as the performance criteria. The simulation study and the real data example results show that the proposed MJPR method outperforms the Poisson ridge regression, Jackknifed Poisson ridge regression and the ML in all of the different situations evaluated in this paper.     

Restricted Two Parameter Ridge Estimator

In 2005 Lipovetsky and Conklin proposed an estimator, the two parameter ridge estimator (TRE), as an alternative to the ordinary least squares estimator (OLSE) and the ordinary ridge estimator (RE) in the presence of multicollinearity, and in 2006 Lipovetsky improved the two parameter model. In this paper, we introduce two new estimators, one of which is the modified two parameter ridge estimator (MTRE) defined by following Swindel's paper of 1976. The other one is the restricted two parameter ridge estimator (RTRE) which is derived by setting additional linear restrictions on the parameter vectors. This estimator is a generalization of the restricted least squares estimator (RLSE) and includes the restricted ridge estimator (RRE) proposed by Groß in 2003. A numerical example is provided and a simulation study is conducted for the comparisons of the RTRE with the OLSE, RLSE, RE, RRE and TRE.     

Two kinds of restricted modified estimators in linear regression model

In this paper, we introduce two kinds of new restricted estimators called restricted modified Liu estimator and restricted modified ridge estimator based on prior information for the vector of parameters in a linear regression model with linear restrictions. Furthermore, the performance of the proposed estimators in mean squares error matrix sense is derived and compared. Finally, a numerical example and a Monte Carlo simulation are given to illustrate some of the theoretical results.     

Some ridge regression estimators for the zero-inflated Poisson model

The zero-inflated Poisson regression model is commonly used when analyzing economic data that come in the form of non-negative integers since it accounts for excess zeros and overdispersion of the dependent variable. However, a problem often encountered when analyzing economic data that has not been addressed for this model is multicollinearity. This paper proposes ridge regression (RR) estimators and some methods for estimating the ridge parameter k for a non-negative model. A simulation study has been conducted to compare the performance of the estimators. Both mean squared error and mean absolute error are considered as the performance criteria. The simulation study shows that some estimators are better than the commonly used maximum-likelihood estimator and some other RR estimators. Based on the simulation study and an empirical application, some useful estimators are recommended for practitioners.     

A revised Cholesky decomposition to combat multicollinearity in multiple regression models

As known, the ordinary least-squares estimator (OLSE) is unbiased and also, has the minimum variance among all the linear unbiased estimators. However, under multicollinearity the estimator is generally unstable and poor in the sense that variance of the regression coefficients may be inflated and absolute values of the estimates may be too large. There are several classes of biased estimators in statistical literature to decrease the effect of multicollinearity in the design matrix. Here, based on the Cholesky decomposition, we propose such an estimator which makes the data to be slightly distorted. The exact risk expressions as well as the biases are derived for the proposed estimator. Also, some results demonstrating superiority of the suggested estimator over OLSE are obtained. Finally, a Monté-Carlo simulation study and a real data application related to acetylene data are presented to support our theoretical discussions.     

Good ridge estimators based on prior information

Ridge regression is re-examined and ridge estimators based on prior information are introduced. A necessary and sufficient condition is given for such ridge estimators to yield estimators of every nonnull linear combination of the regression coefficients with smaller mean square error than that of the Gauss-Markov best linear unbiased estimator.     

Choosing shrinkage estimators for regression problems

A Bayesian formulation of the canonical form of the standard regression model is used to compare various Stein-type estimators and the ridge estimator of regression coefficients, A particular (“constant prior”) Stein-type estimator having the same pattern of shrinkage as the ridge estimator is recommended for use.     

A ridge-type estimator and good prior means

Swindel (1976) introduced a modified ridge regression estimator based on prior information. A necessary and sufficient condition is derived for Swindel's proposed estimator to have lower risk than the conventional ordinary ridge regression estimator when both estimators are computed using the same value of k.     

On the ridge regression estimator with sub-space restriction

In the linear regression model with elliptical errors, a shrinkage ridge estimator is proposed. In this regard, the restricted ridge regression estimator under sub-space restriction is improved by incorporating a general function which satisfies Taylor’s series expansion. Approximate quadratic risk function of the proposed shrinkage ridge estimator is evaluated in the elliptical regression model. A Monte Carlo simulation study and analysis based on a real data example are considered for performance analysis. It is evident from the numerical results that the shrinkage ridge estimator performs better than both unrestricted and restricted estimators in the multivariate t-regression model, for some specific cases.     

Feasible robust estimator in restricted semiparametric regression models based on the LTS approach

Under some nonstochastic linear restrictions based on either additional information or prior knowledge in a semiparametric regression model, a family of feasible generalized robust estimators for the regression parameter is proposed. The least trimmed squares (LTS) method proposed by Rousseeuw as a highly robust regression estimator is a statistical technique for fitting a regression model based on the subset of h observations (out of n) whose least-square fit possesses the smallest sum of squared residuals. The coverage h may be set between n/2 and n. The LTS estimator involves computing the hyperplane that minimizes the sum of the smallest h squared residuals. For practical purpose, it is assumed that the covariance matrix of the error term is unknown and thus feasible estimators are replaced. Then, we develop an algorithm for the LTS estimator based on feasible methods. Through the Monte Carlo simulation studies and a real data example, performance of the feasible type of robust estimators is compared with the classical ones in restricted semiparametric regression models.     

Unbiased ridge estimation with prior information and ridge trace

A procedure is illustrated to incorporate prior information in the ridge regression model. Unbiased ridge estimators with prior information are defined and a robust estimate of the ridge parameter k is proposed.     

Performance of some new preliminary test ridge regression estimators and their properties

The problem of estimation of the regression coefficients in a multiple regression model is considered under multicollinearity situation when it is suspected that the regression coefficients may be restricted to a subspace. We present the estimators of the regression coefficients combining the idea of preliminary test and ridge regression methodology. Accordingly, we consider three estimators, namely, the unrestricted ridge regression estimator (URRE), the restricted ridge regression estimator (RRRE), and finally, the preliminary test ridge regression estimator (PTRRE). The biases, variancematrices and mean square errors (mse) of the estimators are derived and compared with the usual estimators. Regions of optimality of the estimators are determined by studying the mse criterion. The conditions of superiority of the estimators over the traditional estimators as in Saleh and Han (1990) and Ali and Saleh (1991) have also been discussed.     

Fractional principal components regression: a general approach to biased estimators

Several biased estimators have been proposed as alternatives to the least squares estimator when multicollinearity is present in the multiple linear regression model. The ridge estimator and the principal components estimator are two techniques that have been proposed for such problems. In this paper the class of fractional principal component estimators is developed for the multiple linear regression model. This class contains many of the biased estimators commonly used to combat multicollinearity. In the fractional principal components framework, two new estimation techniques are introduced. The theoretical performances of the new estimators are evaluated and their small sample properties are compared via simulation with the ridge, generalized ridge and principal components estimators     

A modified ridge m-estimator for linear regression model with multicollinearity and outliers

The ordinary least-square estimators for linear regression analysis with multicollinearity and outliers lead to unfavorable results. In this article, we propose a new robust modified ridge M-estimator (MRME) based on M-estimator (ME) to deal with the combined problem resulting from multicollinearity and outliers in the y-direction. MRME outperforms modified ridge estimator, robust ridge estimator and ME, according to mean squares error criterion. Furthermore, a numerical example and a Monte Carlo simulation experiment are given to illustrate some of the theoretical results.     

On the estimation of reliabilty function in a Weibull lifetime distribution

The estimation of the reliability function of the Weibull lifetime model is considered in the presence of uncertain prior information (not in the form of prior distribution) on the parameter of interest. This information is assumed to be available in some sort of a realistic conjecture. In this article, we focus on how to combine sample and non-sample information together in order to achieve improved estimation performance. Three classes of point estimatiors, namely, the unrestricted estimator, the shrinkage estimator and shrinkage preliminary test estimator (SPTE) are proposed. Their asymptotic biases and mean-squared errors are derived and compared. The relative dominance picture of the estimators is presented. Interestingly, the proposed SPTE dominates the unrestricted estimator in a range that is wider than that of the usual preliminary test estimator. A small-scale simulation experiment is used to examine the small sample properties of the proposed estimators. Our simulation investigations have provided strong evidence that corroborates with asymptotic theory. The suggested estimation methods are applied to a published data set to illustrate the performance of the estimators in a real-life situation.     

The influence function of penalized regression estimators

To perform regression analysis in high dimensions, lasso or ridge estimation are a common choice. However, it has been shown that these methods are not robust to outliers. Therefore, alternatives as penalized M-estimation or the sparse least trimmed squares (LTS) estimator have been proposed. The robustness of these regression methods can be measured with the influence function. It quantifies the effect of infinitesimal perturbations in the data. Furthermore, it can be used to compute the asymptotic variance and the mean-squared error (MSE). In this paper we compute the influence function, the asymptotic variance and the MSE for penalized M-estimators and the sparse LTS estimator. The asymptotic biasedness of the estimators make the calculations non-standard. We show that only M-estimators with a loss function with a bounded derivative are robust against regression outliers. In particular, the lasso has an unbounded influence function.     

Shrinkage ridge regression in partial linear models

ABSTRACT

In this paper, shrinkage ridge estimator and its positive part are defined for the regression coefficient vector in a partial linear model. The differencing approach is used to enjoy the ease of parameter estimation after removing the non parametric part of the model. The exact risk expressions in addition to biases are derived for the estimators under study and the region of optimality of each estimator is exactly determined. The performance of the estimators is evaluated by simulated as well as real data sets.     

Positive-rule stein-type almost unbiased ridge estimator in linear regression model

Abstract

In this article, when it is suspected that regression coefficients may be restricted to a subspace, we discuss the parameter estimation of regression coefficients in a multiple regression model. Then, in order to improve the preliminary test almost ridge estimator, we study the positive-rule Stein-type almost unbiased ridge estimator based on the positive-rule stein-type shrinkage estimator and almost unbiased ridge estimator. After that, quadratic bias and quadratic risk values of the new estimator are derived and compared with some relative estimators. And we also discuss the option of parameter k. Finally, we perform a real data example and a Monte Carlo study to illustrate theoretical results.     

Using improved estimation strategies to combat multicollinearity

In this approach, some generalized ridge estimators are defined based on shrinkage foundation. Completely under the suspicion that some sub-space restrictions may occur, we present the estimators of the regression coefficients combining the idea of preliminary test estimator and Stein-rule estimator with the ridge regression methodology for normal models. Their exact risk expressions in addition to biases are derived and the regions of optimality of the estimators are exactly determined along with some numerical analysis. In this regard, the ridge parameter is determined in different disciplines.