Ordinary least squares and Stein-rule predictions in regression models under inclusion of some superfluous variables

This article considers a misspecified linear regression model in which misspecification relates to the inclusion of some explanatory variables. Assuming the distribution of disturbances to be not necessarily normal, this paper investigates the efficiency properties of predictions arising from ordinary least squares and Stein-rule when the aim is to predict either the actual value or the mean value of the study variable.     

The effects of the proxy information on the iterative Stein-rule estimator of the disturbance variance

In a regression model with proxy variables, we consider the iterative estimator of the disturbance variance to obtain more precise estimates. In the formula of the estimator of the disturbance variance, the estimator is obtained by using Stein-rule (SR) estimator instead of OLS (ordinary least squares) estimator is called Iterative estimator of the disturbance variance. It is shown that, in a regression model with proxy variables the mean square error (MSE) of the iterative estimator of the disturbance variance is greater than the MSE of the disturbance variance related to the OLS estimator under certain conditions.     

Stein-rule estimation in models with a lagged-dependemt variable

There is an extensive literature on the Stein-rule estimation of the parameters in a regression model. An important result in this literature is that the Stein-rule does not dominate the least squares estimator in the lower mean squared error sense when there are two or less regressors in the model. However, we note that not much is known about the Stein-rule estimation in dynamic models. This paper is a modest attempt in this direction and it shows that Stein-rule estimation of the lagged coefficient does not dominate the least square indicating that the result of regresion model goes through in the dynamic case.     

Stein-rule estimation under an extended balanced loss function

This paper extends the balanced loss function to a more general setup. The ordinary least squares estimator (OLSE) and Stein-rule estimator (SRE) are exposed to this general loss function with quadratic loss structure in a linear regression model. Their risks are derived when the disturbances in the linear regression model are not necessarily normally distributed. The dominance of OLSE and SRE over each other and the effect of departure from normality assumption of disturbances on the risk property are studied.     

State space modeling of multiple time series: a comment

This paper provides guidance in choosing k₁ andk₂ of the double k-class (KK) estimator such that it will improve upon both the ordinary least squares (OLS) and Stein-rule (SR) estimators in predictive mean squared error (PMSE). Asymptotic bias and mean squared error (MSE) results are derived for nonnormal and other cases. A simulation compares the KK estimator with the OLS and SR estimators.     

Prediction of response values in linear regression models from replicated experiments

This paper considers the problem of prediction in a linear regression model when data sets are available from replicated experiments. Pooling the data sets for the estimation of regression parameters, we present three predictors — one arising from the least squares method and two stemming from Stein-rule method. Efficiency properties of these predictors are discussed when they are used to predict actual and average values of response variable within/outside the sample. Received: November 17, 1999; revised version: August 10, 2000     

SEPARATE VERSUS SYSTEM METHODS OF STEIN-RULE ESTIMATION IN SEEMINGLY UNRELATED REGRESSION MODELS

ABSTRACT

Despite the sizeable literature associated with the seemingly unrelated regression models, not much is known about the use of Stein-rule estimators in these models. This gap is remedied in this paper, in which two families of Stein-rule estimators in seemingly unrelated regression equations are presented and their large sample asymptotic properties explored and evaluated. One family of estimators uses a shrinkage factor obtained solely from the equation under study while the other has a shrinkage factor based on all the equations of the model. Using a quadratic loss measure and Monte-Carlo sampling experiments, the finite sample risk performance of these estimators is also evaluated and compared with the traditional feasible generalized least squares estimator.     

Risk comparison of the Stein-rule estimator in a linear regression model with omitted relevant regressors and multivariate<Emphasis Type="Italic">t</Emphasis> errors under the Pitman nearness criterion

In this paper we consider a linear regression model with omitted relevant regressors and multivariatet error terms. The explicit formula for the Pitman nearness criterion of the Stein-rule (SR) estimator relative to the ordinary least squares (OLS) estimator is derived. It is shown numerically that the dominance of the SR estimator over the OLS estimator under the Pitman nearness criterion can be extended to the case of the multivariatet error distribution when the specification error is not severe. It is also shown that the dominance of the SR estimator over the OLS estimator cannot be extended to the case of the multivariatet error distribution when the specification error is severe. This research is partially supported by the Grants-in-Aid for 21st Century COE program.     

Robust Sparse Regression with High-Breakdown Value

Penalized least squares estimators are sensitive to the influence of outliers like the ordinary least squares estimator. We propose a sparse regression estimator for robust variable selection and estimation based on a robust initial estimator. It is proven that our estimator has at least the same breakdown value as the initial estimator. Numerical examples are presented to illustrate our method.     

Bootstrapping estimators for the seemingly unrelated regressions model

In this paper we consider the possibility of using the bootstrap to estimate the finite sample variability of feasible generalized least squares and improved estimators applied to the seemingly unrelated regressions model. The improved estimators we employ include members of the Stein-rule family and a hierarchical Bayes estimator proposed by Blattberg and George (1991). Simulation experiments are carried out using several SUR examples as well as a very large example based on the price-promotion model, and data, from marketing research.     

Sparse partial least squares regression for simultaneous dimension reduction and variable selection

Summary.  Partial least squares regression has been an alternative to ordinary least squares for handling multicollinearity in several areas of scientific research since the 1960s. It has recently gained much attention in the analysis of high dimensional genomic data. We show that known asymptotic consistency of the partial least squares estimator for a univariate response does not hold with the very large p and small n paradigm. We derive a similar result for a multivariate response regression with partial least squares. We then propose a sparse partial least squares formulation which aims simultaneously to achieve good predictive performance and variable selection by producing sparse linear combinations of the original predictors. We provide an efficient implementation of sparse partial least squares regression and compare it with well-known variable selection and dimension reduction approaches via simulation experiments. We illustrate the practical utility of sparse partial least squares regression in a joint analysis of gene expression and genomewide binding data.     

A Weighted Least Squares Approach to Levene's Test of Homogeneity of Variance

In 1960 Levene suggested a potentially robust test of homogeneity of variance based on an ordinary least squares analysis of variance of the absolute values of mean-based residuals. Levene's test has since been shown to have inflated levels of significance when based on the F-distribution, and tests a hypothesis other than homogeneity of variance when treatments are unequally replicated, but the incorrect formulation is now standard output in several statistical packages. This paper develops a weighted least squares analysis of variance of the absolute values of both mean-based and median-based residuals. It shows how to adjust the residuals so that tests using the F -statistic focus on homogeneity of variance for both balanced and unbalanced designs. It shows how to modify the F -statistics currently produced by statistical packages so that the distribution of the resultant test statistic is closer to an F-distribution than is currently the case. The weighted least squares approach also produces component mean squares that are unbiased irrespective of which variable is used in Levene's test. To complete this aspect of the investigation the paper derives exact second-order moments of the component sums of squares used in the calculation of the mean-based test statistic. It shows that, for large samples, both ordinary and weighted least squares test statistics are equivalent; however they are over-dispersed compared to an F variable.     

Continuous outcomes stochastically mismeasured as discrete random variables: a maximum likelihood approach

In an economic model of retirement behavior, a continuous dependent variable was required; the variable could only be estimated discretely with error, however. Parameter estimates using this dependent variable and ordinary least squares regression are inefficient. In th is paper, we develop a maximum likelihood procedure which adjusts for this heteroscedasticity.     

Efficiency of least squares estimators in the presence of spatial autocorrelation

The effect of spatial autocorrelation on inferences made using ordinary least squares estimation is considered. It is found, in some cases, that ordinary least squares estimators provide a reasonable alternative to the estimated generalized least squares estimators recommended in the spatial statistics literature. One of the most serious problems in using ordinary least squares is that the usual variance estimators are severely biased when the errors are correlated. An alternative variance estimator that adjusts for any observed correlation is proposed. The need to take autocorrelation into account in variance estimation negates much of the advantage that ordinary least squares estimation has in terms of computational simplicity     

Unbiased estimation of the MSE matrices of improved estimators in linear regression

Stein-rule and other improved estimators have scarcely been used in empirical work. One major reason is that it is not easy to obtain precision measures for these estimators. In this paper, we derive unbiased estimators for both the mean squared error (MSE) and the scaled MSE matrices of a class of Stein-type estimators. Our derivation provides the basis for measuring the estimators' precision and constructing confidence bands. Comparisons are made between these MSE estimators and the least squares covariance estimator. For illustration, the methodology is applied to data on energy consumption.     

Comparison of the Iterative Stein-Rule and the Usual Estimators of the Disturbance Variance Under the Pitman Nearness Criterion in a Linear Regression Model with Proxy Variables

In a linear regression model with proxy variables, the iterative Stein-rule estimator and the usual estimator of the disturbance variance is compared under the Pitman Nearness Criterion. The exact expression of Pitman Nearness probability is derived and numerically evaluated.     

Empirical assessment of the applicability of shrinkage predictors in regression when past and future data differ

Jones and Copas (1986) present theoretical and simulation results on the relative merits of a Stein predictor (Copas, 1983) and the ordinary least squares predictor in the usual linear multiple regression model, when certain distributional properties of the regressor variables arising in the past differ from those for which predictions are to be made. Here, extension is made to the practical situation where the true regression parameters are unknown. A hypothesis testing procedure is developed to help determine which of shrinkage and least squares is preferable in any given instance. This approach is applied to explain some empirical evidence on the comparative merits of the two procedures, recently given by Berk (1984).     

Estimating partially linear panel data models with one-way error components

We consider the problem of estimating a partially linear panel data model whenthe error follows an one-way error components structure. We propose a feasiblesemiparametric generalized least squares (GLS) type estimator for estimating the coefficient of the linear component and show that it is asymptotically more efficient than a semiparametric ordinary least squares (OLS) type estimator. We also discussed the case when the regressor of the parametric component is correlated with the error, and propose an instrumental variable GLS-type semiparametric estimator.     

Measuring the degree of severity of heteroskedasticity and the choice between the ols estimator and the 2sae

This paper dwells on the choice between the ordinary least squares and the estimated generalized least squares estimators when the presence of heteroskedasticity is suspected. Since the estimated generalized least squares estimator does not dominate the ordinary least squares estimator completely over the whole parameter space, it is of interest to the researcher to know in advance whether the degree of severity of heteroskedasticity is such that OLS estimator outperforms the estimated generalized least squares (or 2SAE). Casting the problem in the non-spherical error mold and exploiting the principle underlying the Bayesian pretest estimator, an intuitive non-mathematical procedure is proposed to serve as an aid to the researcher in deciding when to use either the ordinary least squares (OLS) or the estimated generalized least squares (2SAE) estimators.     

Least squares estimation of varying‐coefficient hazard regression with application to breast cancer dose‐intensity data

To enhance modeling flexibility, the authors propose a nonparametric hazard regression model, for which the ordinary and weighted least squares estimation and inference procedures are studied. The proposed model does not assume any parametric specifications on the covariate effects, which is suitable for exploring the nonlinear interactions between covariates, time and some exposure variable. The authors propose the local ordinary and weighted least squares estimators for the varying‐coefficient functions and establish the corresponding asymptotic normality properties. Simulation studies are conducted to empirically examine the finite‐sample performance of the new methods, and a real data example from a recent breast cancer study is used as an illustration. The Canadian Journal of Statistics 37: 659–674; 2009 © 2009 Statistical Society of Canada