Estimating partially linear panel data models with one-way error components

We consider the problem of estimating a partially linear panel data model whenthe error follows an one-way error components structure. We propose a feasiblesemiparametric generalized least squares (GLS) type estimator for estimating the coefficient of the linear component and show that it is asymptotically more efficient than a semiparametric ordinary least squares (OLS) type estimator. We also discussed the case when the regressor of the parametric component is correlated with the error, and propose an instrumental variable GLS-type semiparametric estimator.     

Effects of the estimation of covariance matrix parameters in the generalized multivariate linear model

We Consider the generalized multivariate linear model and assume the covariance matrix of the p x 1 vector of responses on a given individual can be represented in the general linear structure form described by Anderson (1973). The effects of the use of estimates of the parameters of the covariance matrix on the generalized least squares estimator of the regression coefficients and on the prediction of a portion of a future vector, when only the first portion of the vector has been observed, are investigated. Approximations are derived for the covariance matrix of the generalized least squares estimator and for the mean square error matrix of the usual predictor, for the practical case where estimated parameters are used.     

On a generalized stein estimator of regression coefficients

This paper studies a generalized Stein estimator of regression coefficients. The small disturbance approximations for the bias and mean square error matrix of the estimator are derived and a necessary and sufficient condition is obtained for the estimator to dominate the ordinary least squares estimator under the mean square error criterion.     

Measuring the degree of severity of heteroskedasticity and the choice between the ols estimator and the 2sae

This paper dwells on the choice between the ordinary least squares and the estimated generalized least squares estimators when the presence of heteroskedasticity is suspected. Since the estimated generalized least squares estimator does not dominate the ordinary least squares estimator completely over the whole parameter space, it is of interest to the researcher to know in advance whether the degree of severity of heteroskedasticity is such that OLS estimator outperforms the estimated generalized least squares (or 2SAE). Casting the problem in the non-spherical error mold and exploiting the principle underlying the Bayesian pretest estimator, an intuitive non-mathematical procedure is proposed to serve as an aid to the researcher in deciding when to use either the ordinary least squares (OLS) or the estimated generalized least squares (2SAE) estimators.     

Theoretical evaluation of prediction error in linear regression with a bivariate response variable containing missing data

Methods for linear regression with multivariate response variables are well described in statistical literature. In this study we conduct a theoretical evaluation of the expected squared prediction error in bivariate linear regression where one of the response variables contains missing data. We make the assumption of known covariance structure for the error terms. On this basis, we evaluate three well-known estimators: standard ordinary least squares, generalized least squares, and a James–Stein inspired estimator. Theoretical risk functions are worked out for all three estimators to evaluate under which circumstances it is advantageous to take the error covariance structure into account.     

Generalized least squares with misspecified serial correlation structures

Summary. The regression literature contains hundreds of studies on serially correlated disturbances. Most of these studies assume that the structure of the error covariance matrix Ω is known or can be estimated consistently from data. Surprisingly, few studies investigate the properties of estimated generalized least squares (GLS) procedures when the structure of Ω is incorrectly identified and the parameters are inefficiently estimated. We compare the finite sample efficiencies of ordinary least squares (OLS), GLS and incorrect GLS (IGLS) estimators. We also prove new theorems establishing theoretical efficiency bounds for IGLS relative to GLS and OLS. Results from an exhaustive simulation study are used to evaluate the finite sample performance and to demonstrate the robustness of IGLS estimates vis-à-vis OLS and GLS estimates constructed for models with known and estimated (but correctly identified) Ω. Some of our conclusions for finite samples differ from established asymptotic results.     

Properties of Designs in Experiments on Spatial Lattices

ABSTRACT

When spatial variation is present in experiments, it is clearly sensible to use designs with favorable properties under both generalized and ordinary least squares. This will make the statistical analysis more robust to misspecification of the spatial model than would be the case if designs were based solely on generalized least squares. In this article, treatment information is introduced as a way of studying the ordinary least squares properties of designs. The treatment information is separated into orthogonal frequency or polynomial components which are assumed to be independent under the spatial model. The well-known trend-resistant designs are those with no treatment information at the very low order frequency or polynomial components which tend to have the higher variances under the spatial model. Ideally, designs would be chosen with all the treatment information distributed at the higher-order components. However, the results in this article show that there are limits on how much trend resistance can be achieved as there are many constraints on the treatment information. In addition, appropriately chosen Williams squares designs are shown to have favorable properties under both ordinary and generalized least squares. At all times, the ordinary least squares properties of the designs are balanced against the generalized least squares objectives of optimizing neighbor balance.     

Generalized and oxdinary least squakes with estimated and unequal variances

An assumption which is often violated in the application of experimental designs is equality of variances. There are several methods available for estimating the unequal variances. This paper covers incorporating different estimators of the variances with the ordinary least squares and generalized least squares. A Monte Carlo study provides more insight into the behavior of these procedures. For some small sample sizes, the incorporations with the ordinary least squares perform satisfactorily, but with the generalized least squares they do not.     

On generalized least squares estimation of the weibull distribution

A simple estimation procedure, based on the generalized least squares method, for the parameters of the Weibull distribution is described and investigated. Through a simulation study, this estimation technique is compared with maximum likelihood estimation, ordinary least squares estimation, and Menon's estimation procedure; this comparison is based on observed relative efficiencies (that is, the ratio of the Cramer-Rao lower bound to the observed mean squared error). Simulation results are presented for samples of size 25. Among the estimators considered in this simulation study, the generalized least squares estimator was found to be the "best" estimator for the shape parameter and a close competitor to the maximum likelihood estimator of the scale parameter.     

Some conditions for optimality of the almost unbiased estimators of regression coefficients

The purpose of this paper is to examine the asymptotic properties of the operational almost unbiased estimator of regression coefficients which includes almost unbiased ordinary ridge estimator a s a special case. The small distrubance approximations for the bias and mean square error matrix of the estimator are derived. As a consequence, it is proved that, under certain conditions, the estimator is more efficient than a general class of estimators given by Vinod and Ullah (1981). Also it is shown that, if the ordinary ridge estimator (ORE) dominates the ordinary least squares estimator then the almost unbiased ordinary ridge estimator does not dominate ORE under the mean square error criterion.     

Comparisons among regression estimators under the generalized mean square error criterion

It is not always prossible to establish a preference ordering among regression estimators in terms of the generalized mean square error criterion. In the paper, we determine when it is feasible to use this criteion to couduct comparisons among ordinary least squares, principal components, ridge regression, and shrunken least squares estimators.     

On choosing estimators'in a simple linear errors-in-variables model

This paper focuses on studying the accuracy of two well-known estimators in a simple errors-in-variables model, the ordinary least squares and the corrected least squares estimator. As a measure of accuracy of the estimators, the mean squared error is adopted. While Ketellapper (1983) addressed this issue for the case where the error of measurement in the independent variable is known, the present article is concerned with this comparison for the case where the ratio of the error variances is known. Comparison of the mean squared errors of the above estimators leads to a simple rule involving quantities estimable from the data, which can be used for deciding which of the two to be preferred on the basis of higher accuracy.     

Two-Step Residual-Based Estimation of Error Variances for Generalized Least Squares in Split-Plot Experiments

In split-plot experiments, estimation of unknown parameters by generalized least squares (GLS), as opposed to ordinary least squares (OLS), is required, owing to the existence of whole- and subplot errors. However, estimating the error variances is often necessary for GLS. Restricted maximum likelihood (REML) is an established method for estimating the error variances, and its benefits have been highlighted in many previous studies. This article proposes a new two-step residual-based approach for estimating error variances. Results of numerical simulations indicate that the proposed method performs sufficiently well to be considered as a suitable alternative to REML.     

Prediction Error Property of the Lasso Estimator and its Generalization

The lasso procedure is an estimator‐shrinkage and variable selection method. This paper shows that there always exists an interval of tuning parameter values such that the corresponding mean squared prediction error for the lasso estimator is smaller than for the ordinary least squares estimator. For an estimator satisfying some condition such as unbiasedness, the paper defines the corresponding generalized lasso estimator. Its mean squared prediction error is shown to be smaller than that of the estimator for values of the tuning parameter in some interval. This implies that all unbiased estimators are not admissible. Simulation results for five models support the theoretical results.     

On exact small sample properties of the minimax generalized ridge regression estimators

The purpose of this paper is to compare sampling performance of the minimax generalized ridge regression estimators considered by Casella (1985) with that of ordinary least squares estimator by numerical calculations of exact mean squared error of these estimators.     

Estimating the Intercept in an Orthogonally Blocked Experiment when the Block Effects are Random

Abstract

For an orthogonally blocked experiment, Khuri [Khuri, A. I. (1992). Response surface models with random block effects. Technometrics 34:26–37] has shown that the ordinary least squares estimator, the generalized least squares estimator and the intra-block estimator of the factor effects in a response surface model with random block effects coincide. The ordinary least squares estimator ignores the blocks, whereas the generalized least squares and the intra-block estimators treat the block effects as random and fixed, respectively. As shown in this paper, the equivalence does not hold for the estimation of the intercept when the block sizes are heterogeneous. Practical examples are given to illustrate the theoretical results.     

Factor analysis regression

The paper describes two regression models—principal components and maximum-likelihood factor analysis—which may be used when the stochastic predictor varibles are highly intereorrelated and/or contain measurement error. The two problems can occur jointly, for example in social-survey data where the true (but unobserved) covariance matrix can be singular. Departure from singularity of the sample dispersion matrix is then due to measurement error. We first consider the more elementary principal components regression model, where it is shown that it can be derived as a special case of (i) canonical correlation, and (ii) restricted least squares. The second part consists of the more general maximum-likelihood factor-analysis regression model, which is derived from the generalized inverse of the product of two singular matrices. Also, it is proved that factor-analysis regression can be considered as an instrumental variables estimator and therefore does not depend on whether factors have been “properly” identified in terms of substantive behaviour. Consequently the additional task of rotating factors to “simple structure” does not arise.     

Some finite sample properties of generalized ridge regression estimators

In this paper we study the mean square error properties of the generalized ridge estimator. We obtain the exact and the approximate bias and the mean square error of the operational generalized ridge estimator in terms of G( ) functions. We show, among other things, that the operational generalized ridge estimator does not dominate the ordinary least squares estimator up to a certain order of approximation. Finally, we note that the iterative procedures to obtain coverging ridge estimators should be used with caution.     

On the almost unbiased generalized liu estimator and unbiased estimation of the bias and mse

In this paper, we derive the almost unbiased generalized Liu estimator and examine an exact unbiased estimator of the bias and mean squared error of the feasible generalized Liu estimator . We compare the almost unbiased generalized Liu estimator (AUGLE) with the generalized Liu estimator (GLE) and with the ordinary least squares estimator (OLSE).     

AN EXAMINATION OF ESTIMATED RESIDUALS IN A REGRESSION WITH AN INFINITE ORDER PARAMETRIC MODEL

We consider a linear regression with the error term that obeys an autoregressive model of infinite order and estimate parameters of the models. The parameters of the autoregressive model should be estimated based on estimated residuals obtained by means of the method of ordinary least squares, because the errors are unobservable. The consistency of the coefficients, variance and spectral density of the model obeyed by the error term is shown. Further, we estimate the coefficients of the linear regression by means of the method of estimated generalized least squares. We also show the consistency of the estimator.