On the almost unbiased ridge regression estimator

The purpose of this paper is two-fold. One is to compare the almost unbiased generalized ridge regression (AUGRR) estimator proposed by Singh, Chaubey and Dwivedi (1986) with the generalized ridge regression (GRR) estimator and with the ordinary least squares (OLS) estimator in terms of the mean squared error criterion. Second is to examine small sample properties of the operational almost unbiased ordinary ridge regression (AUORR) estimator by Monte Carlo experiments.     

Linearized Restricted Ridge Regression Estimator in Linear Regression

This article primarily aims to put forward the linearized restricted ridge regression (LRRR) estimator in linear regression models. Two types of LRRR estimators are investigated under the PRESS criterion and the optimal LRRR estimators and the optimal restricted generalized ridge regression estimator are obtained. We apply the results to the Hald data and finally make a simulation study by using the method of McDonald and Galarneau.     

A new modified Jackknifed estimator for the Poisson regression model

The Poisson regression is very popular in applied researches when analyzing the count data. However, multicollinearity problem arises for the Poisson regression model when the independent variables are highly intercorrelated. Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators and some methods for estimating the ridge parameter k in the Poisson regression have been proposed. It has been found that some estimators are better than the commonly used maximum-likelihood (ML) estimator and some other RR estimators. In this study, the modified Jackknifed Poisson ridge regression (MJPR) estimator is proposed to remedy the multicollinearity. A simulation study and a real data example are provided to evaluate the performance of estimators. Both mean-squared error and the percentage relative error are considered as the performance criteria. The simulation study and the real data example results show that the proposed MJPR method outperforms the Poisson ridge regression, Jackknifed Poisson ridge regression and the ML in all of the different situations evaluated in this paper.     

Linearized Ridge Regression Estimator in Linear Regression

In this article, we aim to study the linearized ridge regression (LRR) estimator in a linear regression model motivated by the work of Liu (1993). The LRR estimator and the two types of generalized Liu estimators are investigated under the PRESS criterion. The method of obtaining the optimal generalized ridge regression (GRR) estimator is derived from the optimal LRR estimator. We apply the Hald data as a numerical example and then make a simulation study to show the main results. It is concluded that the idea of transforming the GRR estimator as a complicated function of the biasing parameters to a linearized version should be paid more attention in the future.     

Choosing shrinkage estimators for regression problems

A Bayesian formulation of the canonical form of the standard regression model is used to compare various Stein-type estimators and the ridge estimator of regression coefficients, A particular (“constant prior”) Stein-type estimator having the same pattern of shrinkage as the ridge estimator is recommended for use.     

Fractional principal components regression: a general approach to biased estimators

Several biased estimators have been proposed as alternatives to the least squares estimator when multicollinearity is present in the multiple linear regression model. The ridge estimator and the principal components estimator are two techniques that have been proposed for such problems. In this paper the class of fractional principal component estimators is developed for the multiple linear regression model. This class contains many of the biased estimators commonly used to combat multicollinearity. In the fractional principal components framework, two new estimation techniques are introduced. The theoretical performances of the new estimators are evaluated and their small sample properties are compared via simulation with the ridge, generalized ridge and principal components estimators     

Performance of some new preliminary test ridge regression estimators and their properties

The problem of estimation of the regression coefficients in a multiple regression model is considered under multicollinearity situation when it is suspected that the regression coefficients may be restricted to a subspace. We present the estimators of the regression coefficients combining the idea of preliminary test and ridge regression methodology. Accordingly, we consider three estimators, namely, the unrestricted ridge regression estimator (URRE), the restricted ridge regression estimator (RRRE), and finally, the preliminary test ridge regression estimator (PTRRE). The biases, variancematrices and mean square errors (mse) of the estimators are derived and compared with the usual estimators. Regions of optimality of the estimators are determined by studying the mse criterion. The conditions of superiority of the estimators over the traditional estimators as in Saleh and Han (1990) and Ali and Saleh (1991) have also been discussed.     

On the ridge regression estimator with sub-space restriction

In the linear regression model with elliptical errors, a shrinkage ridge estimator is proposed. In this regard, the restricted ridge regression estimator under sub-space restriction is improved by incorporating a general function which satisfies Taylor’s series expansion. Approximate quadratic risk function of the proposed shrinkage ridge estimator is evaluated in the elliptical regression model. A Monte Carlo simulation study and analysis based on a real data example are considered for performance analysis. It is evident from the numerical results that the shrinkage ridge estimator performs better than both unrestricted and restricted estimators in the multivariate t-regression model, for some specific cases.     

MORE ON THE PRE-TEST ESTIMATOR IN RIDGE REGRESSION

ABSTRACT

The problem of estimation of the regression coefficients in a multiple regression model is considered under a multicollinearity situation when it is suspected that the regression coefficients may be restricted to a subspace. The objective of this paper is to compare the usual preliminary test estimator and the preliminary test ridge regression estimator in the sense of the dispersion matrix of one dominating that of the other. In particular we proved two results giving necessary and sufficient conditions for the superiority of the preliminary test ridge regression estimator over the preliminary test estimator associated with the δ = 0 (or Δ = 0) and δ ≠ 0 (or Δ ≠ 0).     

Using Liu-Type Estimator to Combat Collinearity

Abstract

Linear regression model and least squares method are widely used in many fields of natural and social sciences. In the presence of collinearity, the least squares estimator is unstable and often gives misleading information. Ridge regression is the most common method to overcome this problem. We find that when there exists severe collinearity, the shrinkage parameter selected by existing methods for ridge regression may not fully address the ill conditioning problem. To solve this problem, we propose a new two-parameter estimator. We show using both theoretic results and simulation that our new estimator has two advantages over ridge regression. First, our estimator has less mean squared error (MSE). Second, our estimator can fully address the ill conditioning problem. A numerical example from literature is used to illustrate the results.     

Cox regression analysis in presence of collinearity: an application to assessment of health risks associated with occupational radiation exposure

This paper considers the analysis of time to event data in the presence of collinearity between covariates. In linear and logistic regression models, the ridge regression estimator has been applied as an alternative to the maximum likelihood estimator in the presence of collinearity. The advantage of the ridge regression estimator over the usual maximum likelihood estimator is that the former often has a smaller total mean square error and is thus more precise. In this paper, we generalized this approach for addressing collinearity to the Cox proportional hazards model. Simulation studies were conducted to evaluate the performance of the ridge regression estimator. Our approach was motivated by an occupational radiation study conducted at Oak Ridge National Laboratory to evaluate health risks associated with occupational radiation exposure in which the exposure tends to be correlated with possible confounders such as years of exposure and attained age. We applied the proposed methods to this study to evaluate the association of radiation exposure with all-cause mortality.     

A new estimator combining the ridge regression and the restricted least squares methods of estimation

It is well-known in the literature on multicollinearity that one of the major consequences of multicollinearity on the ordinary least squares estimator is that the estimator produces large sampling variances, which in turn might inappropriately lead to exclusion of otherwise significant coefficients from the model. To circumvent this problem, two accepted estimation procedures which are often suggested are the restricted least squares method and the ridge regression method. While the former leads to a reduction in the sampling variance of the estimator, the later ensures a smaller mean square error value for the estimator. In this paper we have proposed a new estimator which is based on a criterion that combines the ideas underlying these two estimators. The standard properties of this new estimator have been studied in the paper. It has also been shown that this estimator is superior to both the restricted least squares as well as the ordinary ridge regression estimators by the criterion of mean sauare error of the estimator of the regression coefficients when the restrictions are indeed correct. The conditions for superiority of this estimator over the other two have also been derived for the situation when the restrictions are not correct.     

THE EXAMINATION AND ANALYSIS OF RESIDUALS FOR SOME BIASED ESTIMATORS IN LINEAR REGRESSION

In the presence of collinearity certain biased estimation procedures like ridge regression, generalized inverse estimator, principal component regression, Liu estimator, or improved ridge and Liu estimators are used to improve the ordinary least squares (OLS) estimates in the linear regression model. In this paper new biased estimator (Liu estimator), almost unbiased (improved) Liu estimator and their residuals will be analyzed and compared with OLS residuals in terms of mean-squared error.     

Robust ridge and robust Liu estimator for regression based on the LTS estimator

In the multiple linear regression analysis, the ridge regression estimator and the Liu estimator are often used to address multicollinearity. Besides multicollinearity, outliers are also a problem in the multiple linear regression analysis. We propose new biased estimators based on the least trimmed squares (LTS) ridge estimator and the LTS Liu estimator in the case of the presence of both outliers and multicollinearity. For this purpose, a simulation study is conducted in order to see the difference between the robust ridge estimator and the robust Liu estimator in terms of their effectiveness; the mean square error. In our simulations, the behavior of the new biased estimators is examined for types of outliers: X-space outlier, Y-space outlier, and X-and Y-space outlier. The results for a number of different illustrative cases are presented. This paper also provides the results for the robust ridge regression and robust Liu estimators based on a real-life data set combining the problem of multicollinearity and outliers.     

A new biased estimator based on ridge estimation

In this paper we introduce a new biased estimator for the vector of parameters in a linear regression model and discuss its properties. We show that our new biased estimator is superior, in the mean square error(mse) sense, to the ordinary least squares (OLS) estimator, the ordinary ridge regression (ORR) estimator and the Liu estimator. We also compare the performance of our new biased estimator with two other special Liu-type estimators proposed in Liu (2003). We illustrate our findings with a numerical example based on the widely analysed dataset on Portland cement.     

Rank-based ridge estimation in multiple linear regression

Multicollinearity and model misspecification are frequently encountered problems in practice that produce undesirable effects on classical ordinary least squares (OLS) regression estimator. The ridge regression estimator is an important tool to reduce the effects of multicollinearity, but it is still sensitive to a model misspecification of error distribution. Although rank-based statistical inference has desirable robustness properties compared to the OLS procedures, it can be unstable in the presence of multicollinearity. This paper introduces a rank regression estimator for regression parameters and develops tests for general linear hypotheses in a multiple linear regression model. The proposed estimator and the tests have desirable robustness features against the multicollinearity and model misspecification of error distribution. Asymptotic behaviours of the proposed estimator and the test statistics are investigated. Real and simulated data sets are used to demonstrate the feasibility and the performance of the estimator and the tests.     

Ridge Regression Estimation for Survey Samples

This paper describes procedure for constructing a vector of regression weights. Under the regression superpopulation model, the ridge regression estimator that has minimum model mean squared error is derived. Through a simulation study, we compare the ridge regression weights, regression weights, quadratic programming weights, and raking ratio weights. The ridge regression procedure with weights bounded by zero performed very well.     

Distribution and density functions of the feasible generalized ridge regression estimator

In this paper, we derive the distribution and density functions of the feasible generalized ridge regression (GRR) estimator. It is shown that when the absolute value of a regression coefficient is close to zero, the distribution of the feasible GRR estimator is bimodal and has thinner tails than that of the OLS estimator.     

Improved robust ridge M-estimation

It is developed that non-sample prior information about regression vector-parameter, usually in the form of constraints, improves the risk performance of the ordinary least squares estimator (OLSE) when it is shrunken. However, in practice, it may happen that both multicollinearity and outliers exist simultaneously in the data. In such a situation, the use of robust ridge estimator is suggested to overcome the undesirable effects of the OLSE. In this article, some prior information in the form of constraints is employed to improve the performance of this estimator in the multiple regression model. In this regard, shrinkage ridge robust estimators are defined. Advantages of the proposed estimators over the usual robust ridge estimator are also investigated using Monte-Carlo simulation as well as a real data example.     

Ridge regression estimators in the linear regression models with non-spherical errors

The present paper considers a family of ordinary ridge regression estimators in the linear regression model when the disturbances covariance matrix depends upon a few unknown parameters. An asymptotic expansion for the distribution of the ridge regression estimator is developed and under the quadratic loss function its asymptotic risk is compared with that of the feasible GLS estimator.