Comparisons of the <Emphasis Type="Italic">r</Emphasis> − <Emphasis Type="Italic">k</Emphasis> class estimator to the ordinary least squares estimator under the Pitman’s closeness criterion

In the presence of multicollinearity, the r − k class estimator is proposed as an alternative to the ordinary least squares (OLS) estimator which is a general estimator including the ordinary ridge regression (ORR), the principal components regression (PCR) and the OLS estimators. Comparison of competing estimators of a parameter in the sense of mean square error (MSE) criterion is of central interest. An alternative criterion to the MSE criterion is the Pitman’s (1937) closeness (PC) criterion. In this paper, we compare the r − k class estimator to the OLS estimator in terms of PC criterion so that we can get the comparison of the ORR estimator to the OLS estimator under the PC criterion which was done by Mason et al. (1990) and also the comparison of the PCR estimator to the OLS estimator by means of the PC criterion which was done by Lin and Wei (2002).     

Finite sample properties of an HPT estimator when each individual regression coefficient is estimated in a misspecified linear regression model

ABSTRACT

In this paper, assuming that there exist omitted variables in the specified model, we analytically derive the exact formula for the mean squared error (MSE) of a heterogeneous pre-test (HPT) estimator whose components are the ordinary least squares (OLS) and feasible ridge regression (FRR) estimators. Since we cannot examine the MSE performance analytically, we execute numerical evaluations to investigate small sample properties of the HPT estimator, and compare the MSE performance of the HPT estimator with those of the FRR estimator and the usual OLS estimator. Our numerical results show that (1) the HPT estimator is more efficient when the model misspecification is severe; (2) the HPT estimator with the optimal critical value obtained under the correctly specified model can be safely used even when there exist omitted variables in the specified model.     

A Tobit Ridge Regression Estimator

This article analyzes the effects of multicollienarity on the maximum likelihood (ML) estimator for the Tobit regression model. Furthermore, a ridge regression (RR) estimator is proposed since the mean squared error (MSE) of ML becomes inflated when the regressors are collinear. To investigate the performance of the traditional ML and the RR approaches we use Monte Carlo simulations where the MSE is used as performance criteria. The simulated results indicate that the RR approach should always be preferred to the ML estimation method.     

THE EXAMINATION AND ANALYSIS OF RESIDUALS FOR SOME BIASED ESTIMATORS IN LINEAR REGRESSION

In the presence of collinearity certain biased estimation procedures like ridge regression, generalized inverse estimator, principal component regression, Liu estimator, or improved ridge and Liu estimators are used to improve the ordinary least squares (OLS) estimates in the linear regression model. In this paper new biased estimator (Liu estimator), almost unbiased (improved) Liu estimator and their residuals will be analyzed and compared with OLS residuals in terms of mean-squared error.     

The relative efficiency of the restricted estimators in linear regression models

This paper deals with the problem of multicollinearity in a multiple linear regression model with linear equality restrictions. The restricted two parameter estimator which was proposed in case of multicollinearity satisfies the restrictions. The performance of the restricted two parameter estimator over the restricted least squares (RLS) estimator and the ordinary least squares (OLS) estimator is examined under the mean square error (MSE) matrix criterion when the restrictions are correct and not correct. The necessary and sufficient conditions for the restricted ridge regression, restricted Liu and restricted shrunken estimators, which are the special cases of the restricted two parameter estimator, to have a smaller MSE matrix than the RLS and the OLS estimators are derived when the restrictions hold true and do not hold true. Theoretical results are illustrated with numerical examples based on Webster, Gunst and Mason data and Gorman and Toman data. We conduct a final demonstration of the performance of the estimators by running a Monte Carlo simulation which shows that when the variance of the error term and the correlation between the explanatory variables are large, the restricted two parameter estimator performs better than the RLS estimator and the OLS estimator under the configurations examined.     

Exact small sample properties of an operational variant of the minimum mean squared error estimator

In this paper, we show a sufficient condition for an operational variant of the minimum mean squared error estimator (simply, the minimum MSE estimator) to dominate the ordinary least squares (OLS) estimator. It is also shown numerically that the minimum MSE estimator dominates the OLS estimator if the number of regression coefficients is larger than or equal to three, even if the sufficient condition is not satisfied. When the number of regression coefficients is smaller than three, our numerical results show that the gain in MSE of using the minimum MSE estimator is larger than the loss.     

Two-stage estimation of inequality-constrained marginal linear models with longitudinal data

Inequality-constrained regression models have received increased attention in longitudinal analysis during recent years. Regression parameters are usually obtained from iteration algorithms. An analytical formulae of the estimators cannot be provided. Therefore, the asymptotic behavior of estimators has not been fully clarified yet. This paper presents a TS estimation (TS for short) and the asymptotic distribution of the estimators. Simulations are conducted to compare constrained TS estimation, constrained ordinary least squares (OLS) estimation and TS estimation in terms of sample bias, sample mean-square error (MSE) and sample variance of the estimators.     

A class of generalized ridge estimators

Presence of collinearity among the explanatory variables results in larger standard errors of parameters estimated. When multicollinearity is present among the explanatory variables, the ordinary least-square (OLS) estimators tend to be unstable due to larger variance of the estimators of the regression coefficients. As alternatives to OLS estimators few ridge estimators are available in the literature. This article presents some of the popular ridge estimators and attempts to provide (i) a generalized class of ridge estimators and (ii) a modified ridge estimator. The performance of the proposed estimators is investigated with the help of Monte Carlo simulation technique. Simulation results indicate that the suggested estimators perform better than the ordinary least-square (OLS) estimators and other estimators considered in this article.     

MSE performance of the weighted average estimators consisting of shrinkage estimators when each individual regression coefficient is estimated

In this paper, we analytically derive the exact formula for the mean squared error (MSE) of two weighted average (WA) estimators for each individual regression coefficient. Further, we execute numerical evaluations to investigate small sample properties of the WA estimators, and compare the MSE performance of the WA estimators with the other shrinkage estimators and the usual OLS estimator. Our numerical results show that (1) the WA estimators have smaller MSE than the other shrinkage estimators and the OLS estimator over a wide region of parameter space; (2) the range where the relative MSE of the WA estimator is smaller than that of the OLS estimator gets narrower as the number of explanatory variables k increases.     

Minimum mean squared error estimation of each individual coefficient in a linear regression model

In this paper, we derive the exact mean squared error (MSE) of the minimum MSE estimator for each individual coefficient in a linear regression model, and show a sufficient condition for the minimum MSE estimator for each individual coefficient to dominate the OLS estimator. Numerical results show that when the number of independent variables is 2 and 3, the minimum MSE estimator for each individual coefficient can be a good alternative to the OLS and Stein-rule estimators.     

Study of partial least squares and ridge regression methods

This article considers both Partial Least Squares (PLS) and Ridge Regression (RR) methods to combat multicollinearity problem. A simulation study has been conducted to compare their performances with respect to Ordinary Least Squares (OLS). With varying degrees of multicollinearity, it is found that both, PLS and RR, estimators produce significant reductions in the Mean Square Error (MSE) and Prediction Mean Square Error (PMSE) over OLS. However, from the simulation study it is evident that the RR performs better when the error variance is large and the PLS estimator achieves its best results when the model includes more variables. However, the advantage of the ridge regression method over PLS is that it can provide the 95% confidence interval for the regression coefficients while PLS cannot.     

On the almost unbiased ridge regression estimator

The purpose of this paper is two-fold. One is to compare the almost unbiased generalized ridge regression (AUGRR) estimator proposed by Singh, Chaubey and Dwivedi (1986) with the generalized ridge regression (GRR) estimator and with the ordinary least squares (OLS) estimator in terms of the mean squared error criterion. Second is to examine small sample properties of the operational almost unbiased ordinary ridge regression (AUORR) estimator by Monte Carlo experiments.     

The effects of the proxy information on the iterative Stein-rule estimator of the disturbance variance

In a regression model with proxy variables, we consider the iterative estimator of the disturbance variance to obtain more precise estimates. In the formula of the estimator of the disturbance variance, the estimator is obtained by using Stein-rule (SR) estimator instead of OLS (ordinary least squares) estimator is called Iterative estimator of the disturbance variance. It is shown that, in a regression model with proxy variables the mean square error (MSE) of the iterative estimator of the disturbance variance is greater than the MSE of the disturbance variance related to the OLS estimator under certain conditions.     

A Simulation Study of Some Biasing Parameters for the Ridge Type Estimation of Poisson Regression

This article proposes several estimators for estimating the ridge parameter k based on Poisson ridge regression (RR) model. These estimators have been evaluated by means of Monte Carlo simulations. As performance criteria, we have calculated the mean squared error (MSE), the mean value, and the standard deviation of k. The first criterion is commonly used, while the other two have never been used when analyzing Poisson RR. However, these performance criteria are very informative because, if several estimators have an equal estimated MSE, then those with low average value and standard deviation of k should be preferred. Based on the simulated results, we may recommend some biasing parameters that may be useful for the practitioners in the field of health, social, and physical sciences.     

Small sample properties of a ridge regression estimator when there exist omitted variables

In this paper, we derive the exact formulae for moments of the ridge regression estimator proposed by Huang (Econ Lett 62:261–264, 1999), when there exist omitted variables. We show the conditions under which the ridge regression estimator has smaller mean squared error (MSE) than the ordinary least squares estimator. Based on the exact formulae for moments, we compare the bias and MSE performances of both estimators by numerical evaluations.     

On preliminary test ridge regression estimators for linear restrictions in a regression model with non-normal disturbances

In this paper, we study the properties of the preliminary test, restricted and unrestricted ridge regression estimators of the linear regression model with non-normal disturbances. We present the estimators of the regression coefficients combining the idea of preliminary test and ridge regression methodology, when it is suspected that the regression coefficients may be restricted to a subspace and the regression error is distributed as multivariate t. Accordingly we consider three estimators, namely the Unrestricted Ridge Regression Estimator (URRRE), the Restricted Ridge Regression Estimator (RRRE) and finally the Preliminary test Ridge Regression Estimator (PTRRE). The biases and the mean square error (MSE) of the estimators are derived under the null and alternative hypotheses and compared with the usual estimators. By studying the MSE criterion, the regions of optimahty of the estimators are determined.     

Using Liu-Type Estimator to Combat Collinearity

Abstract

Linear regression model and least squares method are widely used in many fields of natural and social sciences. In the presence of collinearity, the least squares estimator is unstable and often gives misleading information. Ridge regression is the most common method to overcome this problem. We find that when there exists severe collinearity, the shrinkage parameter selected by existing methods for ridge regression may not fully address the ill conditioning problem. To solve this problem, we propose a new two-parameter estimator. We show using both theoretic results and simulation that our new estimator has two advantages over ridge regression. First, our estimator has less mean squared error (MSE). Second, our estimator can fully address the ill conditioning problem. A numerical example from literature is used to illustrate the results.     

A Bayesian approach with generalized ridge estimation for high-dimensional regression and testing

This paper adopts a Bayesian strategy for generalized ridge estimation for high-dimensional regression. We also consider significance testing based on the proposed estimator, which is useful for selecting regressors. Both theoretical and simulation studies show that the proposed estimator can simultaneously outperform the ordinary ridge estimator and the LSE in terms of the mean square error (MSE) criterion. The simulation study also demonstrates the competitive MSE performance of our proposal with the Lasso under sparse models. We demonstrate the method using the lung cancer data involving high-dimensional microarrays.     

Performance of Kibria's Method for the Heteroscedastic Ridge Regression Model: Some Monte Carlo Evidence

It is common for a linear regression model that the error terms display some form of heteroscedasticity and at the same time, the regressors are also linearly correlated. Both of these problems have serious impact on the ordinary least squares (OLS) estimates. In the presence of heteroscedasticity, the OLS estimator becomes inefficient and the similar adverse impact can also be found on the ridge regression estimator that is alternatively used to cope with the problem of multicollinearity. In the available literature, the adaptive estimator has been established to be more efficient than the OLS estimator when there is heteroscedasticity of unknown form. The present article proposes the similar adaptation for the ridge regression setting with an attempt to have more efficient estimator. Our numerical results, based on the Monte Carlo simulations, provide very attractive performance of the proposed estimator in terms of efficiency. Three different existing methods have been used for the selection of biasing parameter. Moreover, three different distributions of the error term have been studied to evaluate the proposed estimator and these are normal, Student's t and F distribution.