Properties of the ordinary least squares and stein-rule predictions in linear regression models with proxy variables

This article considers a linear regression model in which misspecification relates to the use of a stochastic proxy variable. The analysis indicates the decline in efficiency of the predictions arising from the ordinary least squares and the Stein-rule estimation procedures when a proxy variable is used in the place of an unobservable variable. However, the performance of the Stein-rule predictions is still found to be better than the ordinary least squares predictions over a broad range of k, the characterizing scalar of the Stein-rule estimator.     

SEPARATE VERSUS SYSTEM METHODS OF STEIN-RULE ESTIMATION IN SEEMINGLY UNRELATED REGRESSION MODELS

ABSTRACT

Despite the sizeable literature associated with the seemingly unrelated regression models, not much is known about the use of Stein-rule estimators in these models. This gap is remedied in this paper, in which two families of Stein-rule estimators in seemingly unrelated regression equations are presented and their large sample asymptotic properties explored and evaluated. One family of estimators uses a shrinkage factor obtained solely from the equation under study while the other has a shrinkage factor based on all the equations of the model. Using a quadratic loss measure and Monte-Carlo sampling experiments, the finite sample risk performance of these estimators is also evaluated and compared with the traditional feasible generalized least squares estimator.     

Minimum mean squared error estimation of each individual coefficient in a linear regression model

In this paper, we derive the exact mean squared error (MSE) of the minimum MSE estimator for each individual coefficient in a linear regression model, and show a sufficient condition for the minimum MSE estimator for each individual coefficient to dominate the OLS estimator. Numerical results show that when the number of independent variables is 2 and 3, the minimum MSE estimator for each individual coefficient can be a good alternative to the OLS and Stein-rule estimators.     

Application of Stein Rules to Combination Forecasting

We propose some Stein-rule combination forecasting methods that are designed to ameliorate the estimation risk inherent in making operational the variance–covariance method for constructing combination weights. By Monte Carlo simulation, it is shown that this amelioration can be substantial in many cases. Moreover, generalized Stein-rule combinations are proposed that offer the user the opportunity to enhance combination forecasting performance when shrinking the feasible variance–covariance weights toward a fortuitous shrinkage point. In an empirical exercise, the proposed Stein-rule combinations performed well relative to competing combination methods.     

Stein-rule estimator under inclusion of superfluous variables in linear regression models

Stein-rule estimation is a well-known method to improve the unbiased OLSE in the sense of smaller Mean-Square-Error. The paper is investigating the behaviour of this efficiency relation in case of misspecification of the linear model caused by inclusion of superfluous variables     

Prediction of response values in linear regression models from replicated experiments

This paper considers the problem of prediction in a linear regression model when data sets are available from replicated experiments. Pooling the data sets for the estimation of regression parameters, we present three predictors — one arising from the least squares method and two stemming from Stein-rule method. Efficiency properties of these predictors are discussed when they are used to predict actual and average values of response variable within/outside the sample. Received: November 17, 1999; revised version: August 10, 2000     

Ordinary least squares and Stein-rule predictions in regression models under inclusion of some superfluous variables

This article considers a misspecified linear regression model in which misspecification relates to the inclusion of some explanatory variables. Assuming the distribution of disturbances to be not necessarily normal, this paper investigates the efficiency properties of predictions arising from ordinary least squares and Stein-rule when the aim is to predict either the actual value or the mean value of the study variable.     

Stein-rule estimation under an extended balanced loss function

This paper extends the balanced loss function to a more general setup. The ordinary least squares estimator (OLSE) and Stein-rule estimator (SRE) are exposed to this general loss function with quadratic loss structure in a linear regression model. Their risks are derived when the disturbances in the linear regression model are not necessarily normally distributed. The dominance of OLSE and SRE over each other and the effect of departure from normality assumption of disturbances on the risk property are studied.     

Estimation of a linear model under microaggregation by individual ranking

Summary Microaggregation by individual ranking is one of themost commonly applied disclosure control techniques for continuous microdata. The paper studies the effect of microaggregation by individual ranking on the least squares estimation of a multiple linear regression model. It is shown that the traditional least squares estimates are asymptotically unbiased. Moreover, the least squares estimates asymptotically have the same variances as the least squares estimates based on the original (non-aggregated) data. Thus, asymptotically, microaggregation by individual ranking does not result in a loss of efficiency in the least squares estimation of a multiple linear regression model. I thank Hans Schneeweiss for very helpful discussions and comments. Financial support from the Deutsche Forschungsgemeinschaft (German Science Foundation) is gratefully acknowledged.     

Pooling multivariate data

In case it is doubtful whether two sets of data have the same mean vector, four estimation strategies have been developed for the target mean vector. In this situation, the estimates based on a preliminary test as well as on Stein-rule are advantageous. Two measures of relative efficiency are considered; one based on thequadratic loss function, and the other on the determinant of the mean square error matrix. A max-min rule for the size of the preliminary test of significance is presented. It is demonstrated that the shrinkage estimator dominates the classical estimator, whereas none of the shrinkage estimator and the preliminary test estimator dominate each other. The range in the parameter space where preliminary test estimator dominates shrinkage is investigated analytically and computationally. It is found that the shrinkage estimator outperform the preliminary test estimator except in a region around the null hypothesis. Moreover, for large values of a, the level of statistical significance, shrinkage estimator dominates the preliminary test estimator uniformly. The relative dominance of the estimators is presented.     

On characterizations of pitman closeness of some shrinkage estimators

For the multivariate normal mean (vector) estimation problem, some characterizations of the Pitman closest property of a general class of shrinkage (or Stein-rule) estimators (including the so called positive-rule versions) are studied. Further, for the same model when the parameter is restricted to a positively homogeneous cone, Pitman closeness of restricted shrinkage maximum likelihood estimators is established.     

Penalized Weighted Least Squares to Small Area Estimation

In this paper, a penalized weighted least squares approach is proposed for small area estimation under the unit level model. The new method not only unifies the traditional empirical best linear unbiased prediction that does not take sampling design into account and the pseudo‐empirical best linear unbiased prediction that incorporates sampling weights but also has the desirable robustness property to model misspecification compared with existing methods. The empirical small area estimator is given, and the corresponding second‐order approximation to mean squared error estimator is derived. Numerical comparisons based on synthetic and real data sets show superior performance of the proposed method to currently available estimators in the literature.     

Bootstrapping estimators for the seemingly unrelated regressions model

In this paper we consider the possibility of using the bootstrap to estimate the finite sample variability of feasible generalized least squares and improved estimators applied to the seemingly unrelated regressions model. The improved estimators we employ include members of the Stein-rule family and a hierarchical Bayes estimator proposed by Blattberg and George (1991). Simulation experiments are carried out using several SUR examples as well as a very large example based on the price-promotion model, and data, from marketing research.     

MSE performance of the weighted average estimators consisting of shrinkage estimators

In this paper, we consider a regression model and propose estimators which are the weighted averages of two estimators among three estimators; the Stein-rule (SR), the minimum mean squared error (MMSE), and the adjusted minimum mean-squared error (AMMSE) estimators. It is shown that one of the proposed estimators has smaller mean-squared error (MSE) than the positive-part Stein-rule (PSR) estimator over a moderate region of parameter space when the number of the regression coefficients is small (i.e., 3), and its MSE performance is comparable to the PSR estimator even when the number of the regression coefficients is not so small.     

Unbiased estimation of the MSE matrices of improved estimators in linear regression

Stein-rule and other improved estimators have scarcely been used in empirical work. One major reason is that it is not easy to obtain precision measures for these estimators. In this paper, we derive unbiased estimators for both the mean squared error (MSE) and the scaled MSE matrices of a class of Stein-type estimators. Our derivation provides the basis for measuring the estimators' precision and constructing confidence bands. Comparisons are made between these MSE estimators and the least squares covariance estimator. For illustration, the methodology is applied to data on energy consumption.     

The effects of the proxy information on the iterative Stein-rule estimator of the disturbance variance

In a regression model with proxy variables, we consider the iterative estimator of the disturbance variance to obtain more precise estimates. In the formula of the estimator of the disturbance variance, the estimator is obtained by using Stein-rule (SR) estimator instead of OLS (ordinary least squares) estimator is called Iterative estimator of the disturbance variance. It is shown that, in a regression model with proxy variables the mean square error (MSE) of the iterative estimator of the disturbance variance is greater than the MSE of the disturbance variance related to the OLS estimator under certain conditions.     

Estimating coefficients of single-index regression models by minimizing variation

This paper proposes a novel estimation of coefficients in single-index regression models. Unlike the traditional average derivative estimation [Powell JL, Stock JH, Stoker TM. Semiparametric estimation of index coefficients. Econometrica. 1989;57(6):1403–1430; Hardle W, Thomas M. Investigating smooth multiple regression by the method of average derivatives. J Amer Statist Assoc. 1989;84(408):986–995] and semiparametric least squares estimation [Ichimura H. Semiparametric least squares (sls) and weighted sls estimation of single-index models. J Econometrics. 1993;58(1):71–120; Hardle W, Hall P, Ichimura H. Optimal smoothing in single-index models. Ann Statist. 1993;21(1):157–178], the procedure developed in this paper is to estimate the coefficients directly by minimizing the mean variation function and does not involve estimating the link function nonparametrically. As a result, it avoids the selection of the bandwidth or the number of knots, and its implementation is more robust and easier. The resultant estimator is shown to be consistent. Numerical results and real data analysis also show that the proposed procedure is more applicable against model free assumptions.     

Seemingly unrelated regressions with covariance matrix of cross-equation ridge regression residuals

Generalized least squares estimation of a system of seemingly unrelated regressions is usually a two-stage method: (1) estimation of cross-equation covariance matrix from ordinary least squares residuals for transforming data, and (2) application of least squares on transformed data. In presence of multicollinearity problem, conventionally ridge regression is applied at stage 2. We investigate the usage of ridge residuals at stage 1, and show analytically that the covariance matrix based on the least squares residuals does not always result in more efficient estimator. A simulation study and an application to a system of firms' gross investment support our finding.     

Variance estimation based on blocked 3×2 cross-validation in high-dimensional linear regression

In high-dimensional linear regression, the dimension of variables is always greater than the sample size. In this situation, the traditional variance estimation technique based on ordinary least squares constantly exhibits a high bias even under sparsity assumption. One of the major reasons is the high spurious correlation between unobserved realized noise and several predictors. To alleviate this problem, a refitted cross-validation (RCV) method has been proposed in the literature. However, for a complicated model, the RCV exhibits a lower probability that the selected model includes the true model in case of finite samples. This phenomenon may easily result in a large bias of variance estimation. Thus, a model selection method based on the ranks of the frequency of occurrences in six votes from a blocked 3×2 cross-validation is proposed in this study. The proposed method has a considerably larger probability of including the true model in practice than the RCV method. The variance estimation obtained using the model selected by the proposed method also shows a lower bias and a smaller variance. Furthermore, theoretical analysis proves the asymptotic normality property of the proposed variance estimation.     

Risk comparison of the Stein-rule estimator in a linear regression model with omitted relevant regressors and multivariate<Emphasis Type="Italic">t</Emphasis> errors under the Pitman nearness criterion

In this paper we consider a linear regression model with omitted relevant regressors and multivariatet error terms. The explicit formula for the Pitman nearness criterion of the Stein-rule (SR) estimator relative to the ordinary least squares (OLS) estimator is derived. It is shown numerically that the dominance of the SR estimator over the OLS estimator under the Pitman nearness criterion can be extended to the case of the multivariatet error distribution when the specification error is not severe. It is also shown that the dominance of the SR estimator over the OLS estimator cannot be extended to the case of the multivariatet error distribution when the specification error is severe. This research is partially supported by the Grants-in-Aid for 21st Century COE program.