On the restricted almost unbiased estimators in linear regression

In this paper, the restricted almost unbiased ridge regression estimator and restricted almost unbiased Liu estimator are introduced for the vector of parameters in a multiple linear regression model with linear restrictions. The bias, variance matrices and mean square error (MSE) of the proposed estimators are derived and compared. It is shown that the proposed estimators will have smaller quadratic bias but larger variance than the corresponding competitors in literatures. However, they will respectively outperform the latter according to the MSE criterion under certain conditions. Finally, a simulation study and a numerical example are given to illustrate some of the theoretical results.     

The relative efficiency of the restricted estimators in linear regression models

This paper deals with the problem of multicollinearity in a multiple linear regression model with linear equality restrictions. The restricted two parameter estimator which was proposed in case of multicollinearity satisfies the restrictions. The performance of the restricted two parameter estimator over the restricted least squares (RLS) estimator and the ordinary least squares (OLS) estimator is examined under the mean square error (MSE) matrix criterion when the restrictions are correct and not correct. The necessary and sufficient conditions for the restricted ridge regression, restricted Liu and restricted shrunken estimators, which are the special cases of the restricted two parameter estimator, to have a smaller MSE matrix than the RLS and the OLS estimators are derived when the restrictions hold true and do not hold true. Theoretical results are illustrated with numerical examples based on Webster, Gunst and Mason data and Gorman and Toman data. We conduct a final demonstration of the performance of the estimators by running a Monte Carlo simulation which shows that when the variance of the error term and the correlation between the explanatory variables are large, the restricted two parameter estimator performs better than the RLS estimator and the OLS estimator under the configurations examined.     

On the restricted almost unbiased two-parameter estimator in linear regression model

Özkale and Kaçiranlar introduced the restricted two-parameter estimator (RTPE) to deal with the well-known multicollinearity problem in linear regression model. In this paper, the restricted almost unbiased two-parameter estimator (RAUTPE) based on the RTPE is presented. The quadratic bias and mean-squared error of the proposed estimator is discussed and compared with the corresponding competitors in literatures. Furthermore, a numerical example and a Monte Carlo simulation study are given to explain some of the theoretical results.     

Comment on a generalized stochastic restricted ridge regression estimator

In this note, we make some comments about the paper of Alheety and Kibria (2014 Alheety, M.I., Kibria, B.M.G. (2014). A generalized stochastic restricted ridge regression estimator. Commun. Stat. Theor. Meth. 43:4415–4427.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]) and correct the wrongly proved Theorems in that paper.     

Performance of the stein-type two-parameter estimator in multiple linear regression model

In this article, we consider the Stein-type approach to the estimation of the regression parameter in a multiple regression model under a multicollinearity situation. The Stein-type two-parameter estimator is proposed when it is suspected that the regression parameter may be restricted to a subspace. The bias and the quadratic risk of the proposed estimator are derived and compared with the two-parameter estimator (TPE), the restricted TPE and the preliminary test TPE. The conditions of superiority of the proposed estimator are obtained. Finally, a real data example is provided to illustrate some of the theoretical results.     

A Stochastic Restricted Two-Parameter Estimator in Linear Regression Model

Özkale and Kaciranlar (2007 Özakle , M. R. , Kaciranlar , S. ( 2007 ). The restricted and unrestricted two-parameter estimators . Commun. Statist. Theor. Meth. 36 : 2707 – 2725 . [Google Scholar]) proposed a two-parameter estimator (TPE) for the unknown parameter vector in linear regression when exact restrictions are assumed to hold. In this article, under the assumption that the errors are not independent and identically distributed, we introduce a new estimator by combining the ideas underlying the mixed estimator (ME) and the two-parameter estimator when stochastic linear restrictions are assumed to hold. The new estimator is called the stochastic restricted two-parameter estimator (SRTPE) and necessary and sufficient conditions for the superiority of the SRTPE over the ME and TPE are derived by the mean squared error matrix (MSEM) criterion. Furthermore, selection of the biasing parameters is discussed and a numerical example is given to illustrate some of the theoretical results.     

More on the restricted Liu estimator in the logistic regression model

?iray et al. proposed a restricted Liu estimator to overcome multicollinearity in the logistic regression model. They also used a Monte Carlo simulation to study the properties of the restricted Liu estimator. However, they did not present the theoretical result about the mean squared error properties of the restricted estimator compared to MLE, restricted maximum likelihood estimator (RMLE) and Liu estimator. In this article, we compare the restricted Liu estimator with MLE, RMLE and Liu estimator in the mean squared error sense and we also present a method to choose a biasing parameter. Finally, a real data example and a Monte Carlo simulation are conducted to illustrate the benefits of the restricted Liu estimator.     

A non stochastic ridge regression estimator and comparison with the James-Stein estimator

Abstract

This article presents a non-stochastic version of the Generalized Ridge Regression estimator that arises from a discussion of the properties of a Generalized Ridge Regression estimator whose shrinkage parameters are found to be close to their upper bounds. The resulting estimator takes the form of a shrinkage estimator that is superior to both the Ordinary Least Squares estimator and the James-Stein estimator under certain conditions. A numerical study is provided to investigate the range of signal to noise ratio under which the new estimator dominates the James-Stein estimator with respect to the prediction mean square error.     

Feasible robust estimator in restricted semiparametric regression models based on the LTS approach

Under some nonstochastic linear restrictions based on either additional information or prior knowledge in a semiparametric regression model, a family of feasible generalized robust estimators for the regression parameter is proposed. The least trimmed squares (LTS) method proposed by Rousseeuw as a highly robust regression estimator is a statistical technique for fitting a regression model based on the subset of h observations (out of n) whose least-square fit possesses the smallest sum of squared residuals. The coverage h may be set between n/2 and n. The LTS estimator involves computing the hyperplane that minimizes the sum of the smallest h squared residuals. For practical purpose, it is assumed that the covariance matrix of the error term is unknown and thus feasible estimators are replaced. Then, we develop an algorithm for the LTS estimator based on feasible methods. Through the Monte Carlo simulation studies and a real data example, performance of the feasible type of robust estimators is compared with the classical ones in restricted semiparametric regression models.     

Generalized preliminary test stochastic restricted estimator in the linear regression model

ABSTRACT

In this paper, we propose three generalized estimators, namely, generalized unrestricted estimator (GURE), generalized stochastic restricted estimator (GSRE), and generalized preliminary test stochastic restricted estimator (GPTSRE). The GURE can be used to represent the ridge estimator, almost unbiased ridge estimator (AURE), Liu estimator, and almost unbiased Liu estimator. When stochastic restrictions are available in addition to the sample information, the GSRE can be used to represent stochastic mixed ridge estimator, stochastic restricted Liu estimator, stochastic restricted almost unbiased ridge estimator, and stochastic restricted almost unbiased Liu estimator. The GPTSRE can be used to represent the preliminary test estimators based on mixed estimator. Using the GPTSRE, the properties of three other preliminary test estimators, namely preliminary test stochastic mixed ridge estimator, preliminary test stochastic restricted almost unbiased Liu estimator, and preliminary test stochastic restricted almost unbiased ridge estimator can also be discussed. The mean square error matrix criterion is used to obtain the superiority conditions to compare the estimators based on GPTSRE with some biased estimators for the two cases for which the stochastic restrictions are correct, and are not correct. Finally, a numerical example and a Monte Carlo simulation study are done to illustrate the theoretical findings of the proposed estimators.     

Efficient improved estimation of the parameters in two seemingly unrelated regression models

For a system of two seemingly unrelated regression equations, this paper proposes a two-stage covariance improved estimator of the regression coefficients. The new estimator is shown to uniformly dominate the present estimators in terms of generalized mean square error criterion. In addition, we also propose the exact generalized mean square error of new estimator.     

Developing a restricted two-parameter Liu-type estimator: A comparison of restricted estimators in the binary logistic regression model

In the context of estimating regression coefficients of an ill-conditioned binary logistic regression model, we develop a new biased estimator having two parameters for estimating the regression vector parameter β when it is subjected to lie in the linear subspace restriction Hβ = h. The matrix mean squared error and mean squared error (MSE) functions of these newly defined estimators are derived. Moreover, a method to choose the two parameters is proposed. Then, the performance of the proposed estimator is compared to that of the restricted maximum likelihood estimator and some other existing estimators in the sense of MSE via a Monte Carlo simulation study. According to the simulation results, the performance of the estimators depends on the sample size, number of explanatory variables, and degree of correlation. The superiority region of our proposed estimator is identified based on the biasing parameters, numerically. It is concluded that the new estimator is superior to the others in most of the situations considered and it is recommended to the researchers.     

Performance of some new Liu parameters for the linear regression model

Abstract

This article introduces some Liu parameters in the linear regression model based on the work of Shukur, Månsson, and Sjölander. These methods of estimating the Liu parameter d increase the efficiency of Liu estimator. The comparison of proposed Liu parameters and available methods has done using Monte Carlo simulation and a real data set where the mean squared error, mean absolute error and interval estimation are considered as performance criterions. The simulation study shows that under certain conditions the proposed Liu parameters perform quite well as compared to the ordinary least squares estimator and other existing Liu parameters.     

A note on combining ridge and principal component regression

Baye and Parker (1984) proposed the r-k class estimator. The purpose of this note is to deal with the comparisons among the r-k class estimators in terms of the mean square error criterion.     

Consistent estimation approach to tackling collinearity and Berkson-type measurement error in linear regression using adjusted Wald-type estimator

This article proposes a consistent estimation approach in linear regression models for the case when the predictor variables are subject to collinearities and Berkson-type measurement errors simultaneously. Our presented procedure does not rely on ridge regression (RR) methods that have been widely addressed in the literature for ill-conditioned problems resulted from multicollinearity. Instead, we review and propose new consistent estimators due to Wald (1940 Wald, A. (1940). Fitting of straight lines if both variables are subject to error. Ann. Math. Stat. 11:284–300.[Crossref] , [Google Scholar]) so that, except finite fourth moments assumptions, no prior knowledge of parametric settings on observations and errors is used, and there is no need to solve estimating equations for coefficient parameters. The performance of the estimation procedure is compared with that of RR-based estimators by using a variety of numerical experiments through Monte Carlo simulation under estimated mean squared error (EMSE) criterion.     

A two-parameter estimator in the negative binomial regression model

In this article, a two-parameter estimator is proposed to combat multicollinearity in the negative binomial regression model. The proposed two-parameter estimator is a general estimator which includes the maximum likelihood (ML) estimator, the ridge estimator (RE) and the Liu estimator as special cases. Some properties on the asymptotic mean-squared error (MSE) are derived and necessary and sufficient conditions for the superiority of the two-parameter estimator over the ML estimator and sufficient conditions for the superiority of the two-parameter estimator over the RE and the Liu estimator in the asymptotic mean-squared error (MSE) matrix sense are obtained. Furthermore, several methods and three rules for choosing appropriate shrinkage parameters are proposed. Finally, a Monte Carlo simulation study is given to illustrate some of the theoretical results.     

A Bayesian approach with generalized ridge estimation for high-dimensional regression and testing

This paper adopts a Bayesian strategy for generalized ridge estimation for high-dimensional regression. We also consider significance testing based on the proposed estimator, which is useful for selecting regressors. Both theoretical and simulation studies show that the proposed estimator can simultaneously outperform the ordinary ridge estimator and the LSE in terms of the mean square error (MSE) criterion. The simulation study also demonstrates the competitive MSE performance of our proposal with the Lasso under sparse models. We demonstrate the method using the lung cancer data involving high-dimensional microarrays.     

Some conditions for optimality of the almost unbiased estimators of regression coefficients

The purpose of this paper is to examine the asymptotic properties of the operational almost unbiased estimator of regression coefficients which includes almost unbiased ordinary ridge estimator a s a special case. The small distrubance approximations for the bias and mean square error matrix of the estimator are derived. As a consequence, it is proved that, under certain conditions, the estimator is more efficient than a general class of estimators given by Vinod and Ullah (1981). Also it is shown that, if the ordinary ridge estimator (ORE) dominates the ordinary least squares estimator then the almost unbiased ordinary ridge estimator does not dominate ORE under the mean square error criterion.     

Evaluation of the predictive performance of the Liu type estimator

This article discusses the predictive performance of the Liu type (LT) estimator compared to ordinary least squares, principal components, ridge regression, and Liu estimators. The theoretical results are illustrated by a numerical example and a region is established where the LT estimator is uniformly superior to the other mentioned estimators.