Asymptotic approximations to the gain of the 2shi over stein estimators in linear regression models when the disturbances are small

The paper considers a new family of explicit or fully operational two-stage Stein or hierarchial information (2SHI) estimators for linear regression models, and provides an expression for the difference between the risks of these estimators and the usual Stein-rule estimator when the variance of the disturbance is small. The condition under which the 2SHI estimators have smaller average MSE than the Stein-rule estimator is also given.     

Minimum mean squared error estimation of each individual coefficient in a linear regression model

In this paper, we derive the exact mean squared error (MSE) of the minimum MSE estimator for each individual coefficient in a linear regression model, and show a sufficient condition for the minimum MSE estimator for each individual coefficient to dominate the OLS estimator. Numerical results show that when the number of independent variables is 2 and 3, the minimum MSE estimator for each individual coefficient can be a good alternative to the OLS and Stein-rule estimators.     

Shrunken estimators via a pitman nearness criterion

Shrunken estimators have traditionally been developed and studied using mean square error (MSE). Recent research on Pitman nearness (PN), however, indicates that it is an interesting, “intrinsic”, alternative to the mean square error (MSE) criterion for investigating estimators. Thus, we develop a shrunken estimator for the mean of a multivariate normal distribution based on minimizing PN, instead of MSE, Further, since the shrinkage factor of this estimator depends on unknown parameters, we examine two approaches for determining this factor: (1) “plug-in” estimates, (2) a range of values for the factor based on an approximate cońfidence interval for the Pitman Nearness probability. A numerical example is given.     

Exact small sample properties of an operational variant of the minimum mean squared error estimator

In this paper, we show a sufficient condition for an operational variant of the minimum mean squared error estimator (simply, the minimum MSE estimator) to dominate the ordinary least squares (OLS) estimator. It is also shown numerically that the minimum MSE estimator dominates the OLS estimator if the number of regression coefficients is larger than or equal to three, even if the sufficient condition is not satisfied. When the number of regression coefficients is smaller than three, our numerical results show that the gain in MSE of using the minimum MSE estimator is larger than the loss.     

MSE performance of the weighted average estimators consisting of shrinkage estimators when each individual regression coefficient is estimated

In this paper, we analytically derive the exact formula for the mean squared error (MSE) of two weighted average (WA) estimators for each individual regression coefficient. Further, we execute numerical evaluations to investigate small sample properties of the WA estimators, and compare the MSE performance of the WA estimators with the other shrinkage estimators and the usual OLS estimator. Our numerical results show that (1) the WA estimators have smaller MSE than the other shrinkage estimators and the OLS estimator over a wide region of parameter space; (2) the range where the relative MSE of the WA estimator is smaller than that of the OLS estimator gets narrower as the number of explanatory variables k increases.     

Small sample properties of a ridge regression estimator when there exist omitted variables

In this paper, we derive the exact formulae for moments of the ridge regression estimator proposed by Huang (Econ Lett 62:261–264, 1999), when there exist omitted variables. We show the conditions under which the ridge regression estimator has smaller mean squared error (MSE) than the ordinary least squares estimator. Based on the exact formulae for moments, we compare the bias and MSE performances of both estimators by numerical evaluations.     

Mean squared error estimators of small area means using survey weights

Using survey weights, You & Rao [You and Rao, The Canadian Journal of Statistics 2002; 30, 431–439] proposed a pseudo‐empirical best linear unbiased prediction (pseudo‐EBLUP) estimator of a small area mean under a nested error linear regression model. This estimator borrows strength across areas through a linking model, and makes use of survey weights to ensure design consistency and preserve benchmarking property in the sense that the estimators add up to a reliable direct estimator of the mean of a large area covering the small areas. In this article, a second‐order approximation to the mean squared error (MSE) of the pseudo‐EBLUP estimator of a small area mean is derived. Using this approximation, an estimator of MSE that is nearly unbiased is derived; the MSE estimator of You & Rao [You and Rao, The Canadian Journal of Statistics 2002; 30, 431–439] ignored cross‐product terms in the MSE and hence it is biased. Empirical results on the performance of the proposed MSE estimator are also presented. The Canadian Journal of Statistics 38: 598–608; 2010 © 2010 Statistical Society of Canada     

A new estimator of entropy

Vasicek (1976) proposed an estimator of entropy based on spacings. A new estimator of entropy is proposed. This new estimator is based on local linear regression. Comparisons between this new estimator and Vasicek's estimator are made. The mean square error (MSE) of the new estimator is consistently smaller than the MSE of Vasicek's estimator.     

The Statistical Properties of the Equity Estimator

In this article we consider the Equity estimator proposed by Krishnamurthi and Rangaswamy. We show that this estimator is inconsistent and does not necessarily improve on the mean squared error (MSE) of the least squares (LS) estimator. We perform a Monte Carlo experiment based on the price-promotion model used in marketing research, with marketing data, comparing the MSE of the Equity estimator to that of two empirical Bayes estimators and the LS estimator. We find that the empirical Bayes estimators have substantially smaller MSE than the Equity estimator in almost every case.     

Finite sample properties of an HPT estimator when each individual regression coefficient is estimated in a misspecified linear regression model

ABSTRACT

In this paper, assuming that there exist omitted variables in the specified model, we analytically derive the exact formula for the mean squared error (MSE) of a heterogeneous pre-test (HPT) estimator whose components are the ordinary least squares (OLS) and feasible ridge regression (FRR) estimators. Since we cannot examine the MSE performance analytically, we execute numerical evaluations to investigate small sample properties of the HPT estimator, and compare the MSE performance of the HPT estimator with those of the FRR estimator and the usual OLS estimator. Our numerical results show that (1) the HPT estimator is more efficient when the model misspecification is severe; (2) the HPT estimator with the optimal critical value obtained under the correctly specified model can be safely used even when there exist omitted variables in the specified model.     

On an adjustment of degrees of freedom in the minimim mean squared error ertimator

In this paper, we consider an adjustment of degrees of freedom in the minimum mean squared error (MMSE) estimator, We derive the exact MSE of the adjusted MMSE (AMMSE) estimator, and compare the MSE of the AMMSE estimator with those of the Stein-(SR), positive-part Stein-rule (PSR) and MMSE estimators by numerical evaluations. It is shown that the adjustment of degrees of freedom is effective when the noncentrality parameter is close to zero, and the MSE performance of the MMSE estimator can be improved in the wide region of the noncentrality parameter by the adjustment, ft is also shown that the AMMSE estimator can have the smaller MSE than the PSR estimator in the wide region of the noncentrality parameter     

A two-parameter estimator in the negative binomial regression model

In this article, a two-parameter estimator is proposed to combat multicollinearity in the negative binomial regression model. The proposed two-parameter estimator is a general estimator which includes the maximum likelihood (ML) estimator, the ridge estimator (RE) and the Liu estimator as special cases. Some properties on the asymptotic mean-squared error (MSE) are derived and necessary and sufficient conditions for the superiority of the two-parameter estimator over the ML estimator and sufficient conditions for the superiority of the two-parameter estimator over the RE and the Liu estimator in the asymptotic mean-squared error (MSE) matrix sense are obtained. Furthermore, several methods and three rules for choosing appropriate shrinkage parameters are proposed. Finally, a Monte Carlo simulation study is given to illustrate some of the theoretical results.     

On the use of the Stein variance estimator in the double <Emphasis Type="Italic">k</Emphasis>-class estimator when each individual regression coefficient is estimated

In this paper we consider the double k-class estimator which incorporates the Stein variance estimator. This estimator is called the SVKK estimator. We derive the explicit formula for the mean squared error (MSE) of the SVKK estimator for each individual regression coefficient. It is shown analytically that the MSE performance of the Stein-rule estimator for each individual regression coefficient can be improved by utilizing the Stein variance estimator. Also, MSE’s of several estimators included in a family of the SVKK estimators are compared by numerical evaluations.     

The effects of the proxy information on the iterative Stein-rule estimator of the disturbance variance

In a regression model with proxy variables, we consider the iterative estimator of the disturbance variance to obtain more precise estimates. In the formula of the estimator of the disturbance variance, the estimator is obtained by using Stein-rule (SR) estimator instead of OLS (ordinary least squares) estimator is called Iterative estimator of the disturbance variance. It is shown that, in a regression model with proxy variables the mean square error (MSE) of the iterative estimator of the disturbance variance is greater than the MSE of the disturbance variance related to the OLS estimator under certain conditions.     

A pseudo‐empirical best linear unbiased prediction approach to small area estimation using survey weights

The authors develop a small area estimation method using a nested error linear regression model and survey weights. In particular, they propose a pseudo‐empirical best linear unbiased prediction (pseudo‐EBLUP) estimator to estimate small area means. This estimator borrows strength across areas through the model and makes use of the survey weights to preserve the design consistency as the area sample size increases. The proposed estimator also has a nice self‐benchmarking property. The authors also obtain an approximation to the model mean squared error (MSE) of the proposed estimator and a nearly unbiased estimator of MSE. Finally, they compare the proposed estimator with the EBLUP estimator and the pseudo‐EBLUP estimator proposed by Prasad & Rao (1999), using data analyzed earlier by Battese, Harter & Fuller (1988).     

A note on iterative AK composite estimator for Current Population Survey

In this article, we introduce the iterative AK composite estimator for the Current Population Survey. This estimator adopts the AK composite estimator as the initial value and further makes good use of the intrinsic composite scheme of the AK composite estimator. We derive the mean squared error (MSE) formula for the iterative composite estimator and describe how to select the optimal tuning coefficients by minimising the MSE. Finally, we examine the proposed method through a simulation study.     

On variable bandwidth selection in local polynomial regression

The performances of data-driven bandwidth selection procedures in local polynomial regression are investigated by using asymptotic methods and simulation. The bandwidth selection procedures considered are based on minimizing 'prelimit' approximations to the (conditional) mean-squared error (MSE) when the MSE is considered as a function of the bandwidth h . We first consider approximations to the MSE that are based on Taylor expansions around h=0 of the bias part of the MSE. These approximations lead to estimators of the MSE that are accurate only for small bandwidths h . We also consider a bias estimator which instead of using small h approximations to bias naïvely estimates bias as the difference of two local polynomial estimators of different order and we show that this estimator performs well only for moderate to large h . We next define a hybrid bias estimator which equals the Taylor-expansion-based estimator for small h and the difference estimator for moderate to large h . We find that the MSE estimator based on this hybrid bias estimator leads to a bandwidth selection procedure with good asymptotic and, for our Monte Carlo examples, finite sample properties.     

Restricted ridge estimator in generalized linear models: Monte Carlo simulation studies on Poisson and binomial distributed responses

It is known that collinearity among the explanatory variables in generalized linear models (GLMs) inflates the variance of maximum likelihood estimators. To overcome multicollinearity in GLMs, ordinary ridge estimator and restricted estimator were proposed. In this study, a restricted ridge estimator is introduced by unifying the ordinary ridge estimator and the restricted estimator in GLMs and its mean squared error (MSE) properties are discussed. The MSE comparisons are done in the context of first-order approximated estimators. The results are illustrated by a numerical example and two simulation studies are conducted with Poisson and binomial responses.     

MSE performance of the weighted average estimators consisting of shrinkage estimators

In this paper, we consider a regression model and propose estimators which are the weighted averages of two estimators among three estimators; the Stein-rule (SR), the minimum mean squared error (MMSE), and the adjusted minimum mean-squared error (AMMSE) estimators. It is shown that one of the proposed estimators has smaller mean-squared error (MSE) than the positive-part Stein-rule (PSR) estimator over a moderate region of parameter space when the number of the regression coefficients is small (i.e., 3), and its MSE performance is comparable to the PSR estimator even when the number of the regression coefficients is not so small.     

Estimating the variance in nonparametric regression—what is a reasonable choice?

The exact mean-squared error (MSE) of estimators of the variance in nonparametric regression based on quadratic forms is investigated. In particular, two classes of estimators are compared: Hall, Kay and Titterington's optimal difference-based estimators and a class of ordinary difference-based estimators which generalize methods proposed by Rice and Gasser, Sroka and Jennen-Steinmetz. For small sample sizes the MSE of the first estimator is essentially increased by the magnitude of the integrated first two squared derivatives of the regression function. It is shown that in many situations ordinary difference-based estimators are more appropriate for estimating the variance, because they control the bias much better and hence have a much better overall performance. It is also demonstrated that Rice's estimator does not always behave well. Data-driven guidelines are given to select the estimator with the smallest MSE.