A sufficient condition for the MSE dominance of the positive-part shrinkage estimator when each individual regression coefficient is estimated in a misspecified linear regression model

In this paper, assuming that there exist omitted explanatory variables in the specified model, we derive the exact formula for the mean squared error (MSE) of a general family of shrinkage estimators for each individual regression coefficient. It is shown analytically that when our concern is to estimate each individual regression coefficient, the positive-part shrinkage estimators have smaller MSE than the original shrinkage estimators under some conditions even when the relevant regressors are omitted. Also, by numerical evaluations, we showed the effects of our theorem for several specific cases. It is shown that the positive-part shrinkage estimators have smaller MSE than the original shrinkage estimators for wide region of parameter space even when there exist omitted variables in the specified model.     

MSE performance of the weighted average estimators consisting of shrinkage estimators when each individual regression coefficient is estimated

In this paper, we analytically derive the exact formula for the mean squared error (MSE) of two weighted average (WA) estimators for each individual regression coefficient. Further, we execute numerical evaluations to investigate small sample properties of the WA estimators, and compare the MSE performance of the WA estimators with the other shrinkage estimators and the usual OLS estimator. Our numerical results show that (1) the WA estimators have smaller MSE than the other shrinkage estimators and the OLS estimator over a wide region of parameter space; (2) the range where the relative MSE of the WA estimator is smaller than that of the OLS estimator gets narrower as the number of explanatory variables k increases.     

MSE Performance and Minimax Regret Significance Points for a HPT Estimator when each Individual Regression Coefficient is Estimated

In this article, we consider a heterogeneous preliminary test (HPT) estimator whose components are the OLS and feasible ridge regression (FRR) estimators, and derive the exact formulae for the moments of the HPT estimator using mathematical method. Since we cannot examine the MSE of the HPT estimator analytically, we execute the numerical evaluation to investigate the MSE performance of the HPT estimator, and compare the MSE performance of the HPT estimator with those of the FRR estimator and the usual OLS estimator. Furthermore, using the minimax regret criterion proposed by Sawa and Hiromatsu (1973 Sawa , T. , Hiromatsu , T. ( 1973 ). Minimax regret significance points for a preliminary test in regression analysis . Econometrica 41 : 1093 – 1101 .[Crossref], [Web of Science ®] , [Google Scholar]), we derive the optimal critical points of the preliminary F test. Our results show that the optimal significance points are greater than 19% and the optimal signicance points decrease as the denominator degrees of freedom of the preliminary F test statistic increases.     

The relative efficiency of the restricted estimators in linear regression models

This paper deals with the problem of multicollinearity in a multiple linear regression model with linear equality restrictions. The restricted two parameter estimator which was proposed in case of multicollinearity satisfies the restrictions. The performance of the restricted two parameter estimator over the restricted least squares (RLS) estimator and the ordinary least squares (OLS) estimator is examined under the mean square error (MSE) matrix criterion when the restrictions are correct and not correct. The necessary and sufficient conditions for the restricted ridge regression, restricted Liu and restricted shrunken estimators, which are the special cases of the restricted two parameter estimator, to have a smaller MSE matrix than the RLS and the OLS estimators are derived when the restrictions hold true and do not hold true. Theoretical results are illustrated with numerical examples based on Webster, Gunst and Mason data and Gorman and Toman data. We conduct a final demonstration of the performance of the estimators by running a Monte Carlo simulation which shows that when the variance of the error term and the correlation between the explanatory variables are large, the restricted two parameter estimator performs better than the RLS estimator and the OLS estimator under the configurations examined.     

Small sample properties of a ridge regression estimator when there exist omitted variables

In this paper, we derive the exact formulae for moments of the ridge regression estimator proposed by Huang (Econ Lett 62:261–264, 1999), when there exist omitted variables. We show the conditions under which the ridge regression estimator has smaller mean squared error (MSE) than the ordinary least squares estimator. Based on the exact formulae for moments, we compare the bias and MSE performances of both estimators by numerical evaluations.     

PMSE dominance of the positive-part shrinkage estimator in a regression model with proxy variables

Consider a linear regression model with some relevant regressors are unobservable. In such a situation, we estimate the model by using the proxy variables as regressors or by simply omitting the relevant regressors. In this paper, we derive the explicit formula of predictive mean squared error (PMSE) of a general family of shrinkage estimators of regression coefficients. It is shown analytically that the positive-part shrinkage estimator dominates the ordinary shrinkage estimator even when proxy variables are used in place of the unobserved variables. Also, as an example, our result is applied to the double k-class estimator proposed by Ullah and Ullah (Double k-class estimators of coefficients in linear regression. Econometrica. 1978;46:705–722). Our numerical results show that the positive-part double k-class estimator with proxy variables has preferable PMSE performance.     

Minimum mean squared error estimation of each individual coefficient in a linear regression model

In this paper, we derive the exact mean squared error (MSE) of the minimum MSE estimator for each individual coefficient in a linear regression model, and show a sufficient condition for the minimum MSE estimator for each individual coefficient to dominate the OLS estimator. Numerical results show that when the number of independent variables is 2 and 3, the minimum MSE estimator for each individual coefficient can be a good alternative to the OLS and Stein-rule estimators.     

PMSE performance of two different types of preliminary test estimators under a multivariate t error term

Abstract

In this paper, assuming that the error terms follow a multivariate t distribution, we derive the exact formula for the predictive mean squared error (PMSE) of two different types of pretest estimators. It is shown analytically that one of the pretest estimator dominates the SR estimator if a critical value of the pretest is chosen appropriately. Also, we compare the PMSE of the pretest estimators with the MMSE, AMMSE, SR and PSR estimators by numerical evaluations. Our results show that the pretest estimators dominate the OLS estimator for all combinations when the degrees of freedom is not more than 5.     

Comparisons of the <Emphasis Type="Italic">r</Emphasis> − <Emphasis Type="Italic">k</Emphasis> class estimator to the ordinary least squares estimator under the Pitman’s closeness criterion

In the presence of multicollinearity, the r − k class estimator is proposed as an alternative to the ordinary least squares (OLS) estimator which is a general estimator including the ordinary ridge regression (ORR), the principal components regression (PCR) and the OLS estimators. Comparison of competing estimators of a parameter in the sense of mean square error (MSE) criterion is of central interest. An alternative criterion to the MSE criterion is the Pitman’s (1937) closeness (PC) criterion. In this paper, we compare the r − k class estimator to the OLS estimator in terms of PC criterion so that we can get the comparison of the ORR estimator to the OLS estimator under the PC criterion which was done by Mason et al. (1990) and also the comparison of the PCR estimator to the OLS estimator by means of the PC criterion which was done by Lin and Wei (2002).     

Exact small sample properties of an operational variant of the minimum mean squared error estimator

In this paper, we show a sufficient condition for an operational variant of the minimum mean squared error estimator (simply, the minimum MSE estimator) to dominate the ordinary least squares (OLS) estimator. It is also shown numerically that the minimum MSE estimator dominates the OLS estimator if the number of regression coefficients is larger than or equal to three, even if the sufficient condition is not satisfied. When the number of regression coefficients is smaller than three, our numerical results show that the gain in MSE of using the minimum MSE estimator is larger than the loss.     

MSE performance and minimax regret significance points for a HPT estimator under a multivariate t error distribution

In this article, assuming that the error terms follow a multivariate t distribution,we derive the exact formulae forthe moments of the heterogeneous preliminary test (HPT) estimator proposed by Xu (2012b Xu, H. (2012b). MSE performance and minimax regret significance points for a HPT estimator when each individual regression coefficient is estimated. Commun. Stat. Theory Methods 42:2152–2164.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]). We also execute the numerical evaluation to investigate the mean squared error (MSE) performance of the HPT estimator and compare it with those of the feasible ridge regression (FRR) estimator and the usual ordinary least squared (OLS) estimator. Further, we derive the optimal critical values of the preliminary F test for the HPT estimator, using the minimax regret function proposed by Sawa and Hiromatsu (1973 Sawa, T., Hiromatsu, T. (1973). Minimax regret significance points for a preliminary test in regression analysis. Econometrica 41:1093–1101.[Crossref], [Web of Science ®] , [Google Scholar]). Our results show that (1) the optimal significance level (α*) increases as the degrees of freedom of multivariate t distribution (ν₀) increases; (2) when ν₀ ? 10, the value of α* is close to that in the normal error case.     

Performance of ridge estimator in inverse Gaussian regression model

The presence of multicollinearity among the explanatory variables has undesirable effects on the maximum likelihood estimator (MLE). Ridge estimator (RE) is a widely used estimator in overcoming this issue. The RE enjoys the advantage that its mean squared error (MSE) is less than that of MLE. The inverse Gaussian regression (IGR) model is a well-known model in the application when the response variable positively skewed. The purpose of this paper is to derive the RE of the IGR under multicollinearity problem. In addition, the performance of this estimator is investigated under numerous methods for estimating the ridge parameter. Monte Carlo simulation results indicate that the suggested estimator performs better than the MLE estimator in terms of MSE. Furthermore, a real chemometrics dataset application is utilized and the results demonstrate the excellent performance of the suggested estimator when the multicollinearity is present in IGR model.     

On the use of the Stein variance estimator in the double <Emphasis Type="Italic">k</Emphasis>-class estimator when each individual regression coefficient is estimated

In this paper we consider the double k-class estimator which incorporates the Stein variance estimator. This estimator is called the SVKK estimator. We derive the explicit formula for the mean squared error (MSE) of the SVKK estimator for each individual regression coefficient. It is shown analytically that the MSE performance of the Stein-rule estimator for each individual regression coefficient can be improved by utilizing the Stein variance estimator. Also, MSE’s of several estimators included in a family of the SVKK estimators are compared by numerical evaluations.     

Inference on the Common Variance of Correlated Normal Random Variables

ABSTRACT

Scale equivariant estimators of the common variance σ², of correlated normal random variables, have mean squared errors (MSE) which depend on the unknown correlations. For this reason, a scale equivariant estimator of σ² which uniformly minimizes the MSE does not exist. For the equi-correlated case, we have developed three equivariant estimators of σ²: a Bayesian estimator for invariant prior as well as two non-Bayesian estimators. We then generalized these three estimators for the case of several variables with multiple unknown correlations. In addition, we developed a system of confidence intervals which produce the desired coverage probability while being efficient in terms of expected length.     

Using Liu-Type Estimator to Combat Collinearity

Abstract

Linear regression model and least squares method are widely used in many fields of natural and social sciences. In the presence of collinearity, the least squares estimator is unstable and often gives misleading information. Ridge regression is the most common method to overcome this problem. We find that when there exists severe collinearity, the shrinkage parameter selected by existing methods for ridge regression may not fully address the ill conditioning problem. To solve this problem, we propose a new two-parameter estimator. We show using both theoretic results and simulation that our new estimator has two advantages over ridge regression. First, our estimator has less mean squared error (MSE). Second, our estimator can fully address the ill conditioning problem. A numerical example from literature is used to illustrate the results.     

Performance of Kibria's Method for the Heteroscedastic Ridge Regression Model: Some Monte Carlo Evidence

It is common for a linear regression model that the error terms display some form of heteroscedasticity and at the same time, the regressors are also linearly correlated. Both of these problems have serious impact on the ordinary least squares (OLS) estimates. In the presence of heteroscedasticity, the OLS estimator becomes inefficient and the similar adverse impact can also be found on the ridge regression estimator that is alternatively used to cope with the problem of multicollinearity. In the available literature, the adaptive estimator has been established to be more efficient than the OLS estimator when there is heteroscedasticity of unknown form. The present article proposes the similar adaptation for the ridge regression setting with an attempt to have more efficient estimator. Our numerical results, based on the Monte Carlo simulations, provide very attractive performance of the proposed estimator in terms of efficiency. Three different existing methods have been used for the selection of biasing parameter. Moreover, three different distributions of the error term have been studied to evaluate the proposed estimator and these are normal, Student's t and F distribution.     

The density function and the MSE dominance of the pre-test estimator in a heteroscedastic linear regression model with omitted variables

In this paper, we consider a heteroscedastic linear regression model with omitted variables. We derive the density function of the pre-test estimator consisting of the two-stage Aitken estimator (2SAE) and the ordinary least squares estimator (OLSE) after the pre-test for homoscedasticity. We also derive the first two moments based on the density function and show the sufficient condition for the pre-test estimator to dominate the 2SAE in terms of the MSE. Our numerical evaluations show that when this sufficient condition does not hold and when the magnitude of the specification error is large, the pre-test estimator can be dominated by the 2SAE, and further, the 2SAE can be dominated by the OLSE.     

On an adjustment of degrees of freedom in the minimim mean squared error ertimator

In this paper, we consider an adjustment of degrees of freedom in the minimum mean squared error (MMSE) estimator, We derive the exact MSE of the adjusted MMSE (AMMSE) estimator, and compare the MSE of the AMMSE estimator with those of the Stein-(SR), positive-part Stein-rule (PSR) and MMSE estimators by numerical evaluations. It is shown that the adjustment of degrees of freedom is effective when the noncentrality parameter is close to zero, and the MSE performance of the MMSE estimator can be improved in the wide region of the noncentrality parameter by the adjustment, ft is also shown that the AMMSE estimator can have the smaller MSE than the PSR estimator in the wide region of the noncentrality parameter     

The effects of the proxy information on the iterative Stein-rule estimator of the disturbance variance

In a regression model with proxy variables, we consider the iterative estimator of the disturbance variance to obtain more precise estimates. In the formula of the estimator of the disturbance variance, the estimator is obtained by using Stein-rule (SR) estimator instead of OLS (ordinary least squares) estimator is called Iterative estimator of the disturbance variance. It is shown that, in a regression model with proxy variables the mean square error (MSE) of the iterative estimator of the disturbance variance is greater than the MSE of the disturbance variance related to the OLS estimator under certain conditions.     

A generalized inverse regression estimator in multi-univariate linear calibration

A multivariate linear calibration problem, in which response variable is multivariate and explanatory variable is univariate, is considered. In this paper a class of generalized inverse regression estimators is proposed in multi-univariate linear calibration. It includes the classical estimator and the inverse regression one (or Krutchkoff estimator). For the proposed estimator we derive the expressions of bias and mean square error (MSE). Furthermore the behavior of these characteristics is investigated through an analytical method. In addition through a numerical study we confirm the existence of a generalized inverse regression estimator to improve both the classical and the inverse regression estimators on the MSE criterion.