Properties of the ordinary least squares and stein-rule predictions in linear regression models with proxy variables

This article considers a linear regression model in which misspecification relates to the use of a stochastic proxy variable. The analysis indicates the decline in efficiency of the predictions arising from the ordinary least squares and the Stein-rule estimation procedures when a proxy variable is used in the place of an unobservable variable. However, the performance of the Stein-rule predictions is still found to be better than the ordinary least squares predictions over a broad range of k, the characterizing scalar of the Stein-rule estimator.     

Criteria for the equality between ordinary least squares and best linear unbiased estimators under certain linear models

A new necessary and sufficient condition is derived for the equality between the ordinary least-squares estimator and the best linear unbiased estimator of the expectation vector in linear models with certain specific design matrices. This condition is then applied to special cases involving one-way and two-way classification models.     

Bias in the ordinary least squares estimator in the dynamic linear regression model with autocorrelated disturbances

We consider the bias in the Ordinary Least Squares estimator in the linear regression model with a lagged dependent variable as regressor. Results are obtained with independent and auto-correlated disturbances. Asymptotic results are obtained analytically, and finite sample results based on a Monte Carlo study. The substantial biases found suggest the need for an alternative estimator to Ordinary Least Squares and powerful tests for autocorrelated disturbances in the dynamic model.     

Total least squares solution for compositional data using linear models

The restrictive properties of compositional data, that is multivariate data with positive parts that carry only relative information in their components, call for special care to be taken while performing standard statistical methods, for example, regression analysis. Among the special methods suitable for handling this problem is the total least squares procedure (TLS, orthogonal regression, regression with errors in variables, calibration problem), performed after an appropriate log-ratio transformation. The difficulty or even impossibility of deeper statistical analysis (confidence regions, hypotheses testing) using the standard TLS techniques can be overcome by calibration solution based on linear regression. This approach can be combined with standard statistical inference, for example, confidence and prediction regions and bounds, hypotheses testing, etc., suitable for interpretation of results. Here, we deal with the simplest TLS problem where we assume a linear relationship between two errorless measurements of the same object (substance, quantity). We propose an iterative algorithm for estimating the calibration line and also give confidence ellipses for the location of unknown errorless results of measurement. Moreover, illustrative examples from the fields of geology, geochemistry and medicine are included. It is shown that the iterative algorithm converges to the same values as those obtained using the standard TLS techniques. Fitted lines and confidence regions are presented for both original and transformed compositional data. The paper contains basic principles of linear models and addresses many related problems.     

On sample size and precision in ordinary least squares

An expression relating estimation precision in the classical linear model to the number of parameters k and the sample size n is illustrated. A rule of thumb for the sample size is suggested.     

On equality of ordinary least squares estimator,best linear unbiased estimator and best linear unbiased predictor in the general linear model

The equality of ordinary least squares estimator (OLSE), best linear unbiased estimator (BLUE) and best linear unbiased predictor (BLUP) in the general linear model with new observations is investigated through matrix rank method, some new necessary and sufficient conditions are given.     

Relative performance of stein-rule and preliminary test estimators in linear models least squares theory

In the classical (univariare) linear model, bearing the plausibility of a subset of the regression parameters being close to a pivot, shrinkage least squares estimation of the complementary subset is considered. Based on the usual James-Stein rule, shrinkage least squares estimators are constructed, and under an asymptotic setup (allowing the shrinkage parameters to be 'close to ' the pivot), the relative performance of such estimators and the prcliminary test estimators is studied. In this context, the normality of the errors is also avoided under the same asymptotic setup. None of the shrinkage and preliminary test estimators may dominate the other (in the light of the asymptotic distributional risk criterion, as has been developed here), though each of them fares well relative to the classical least squeres estimator. The chice of the shrinkage factor is also examined properly.     

A note on the correlation betweenS 2 and the least squares estimator in the linear regression model

In the following we consider the correlation betweenS ² and the least squares estimator in the linear regression model. We are interested in situations where these two statistics are uncorrelated though the errors are correlated. Conditions are developed without normality assumption, only assuming finite fourth moments of the error distributions. Support by Deutsche Forschungsgemeinschaft Grant No. Tr 253/1-2 is gratefully acknowledged.     

A note on the correlation betweenS 2 and the least squares estimator in the linear regression model

The online version of the original article can be found at     

Bayes least squares linear estimation of densities

Linear, least squares statistical methods in which the "parameters" are interpreted as random variables were introduced by Whittle, and further developed by Hartigan and others. They are applied here to the problem of estimating the coefficients in an orthogonal expansion of a multivariate density, given a simple random sample.     

Asymptotic distribution of least square estimators for linear models with dependent errors

In this paper, we consider the usual linear regression model in the case where the error process is assumed strictly stationary. We use a result from Hannan (Central limit theorems for time series regression. Probab Theory Relat Fields. 1973;26(2):157–170), who proved a Central Limit Theorem for the usual least squares estimator under general conditions on the design and on the error process. Whatever the design satisfying Hannan's conditions, we define an estimator of the covariance matrix and we prove its consistency under very mild conditions. As an application, we show how to modify the usual tests on the linear model in this dependent context, in such a way that the type-I error rate remains asymptotically correct, and we illustrate the performance of this procedure through different sets of simulations.     

Improper use of the ordinary least squares estimator in the switching regression model

The paper considers the consequences of incorrectly using the ordinary least squares estimator, when the true but unknown model is a switching regression. Bias and mean square error express ons are given for slope and residual variance estimators. Except for in very specialized cases the estimators are biased. A numerical exarnple illustrates some of the issues raised and provides a conpelison between the ordinary least squares and maximum likelihood estimators.     

Fitting linear regression models to censored data by least squares and maximum likelihood methods

Several approaches have been suggested for fitting linear regression models to censored data. These include Cox's propor­tional hazard models based on quasi-likelihoods. Methods of fitting based on least squares and maximum likelihoods have also been proposed. The methods proposed so far all require special purpose optimization routines. We describe an approach here which requires only a modified standard least squares routine.

We present methods for fitting a linear regression model to censored data by least squares and method of maximum likelihood. In the least squares method, the censored values are replaced by their expectations, and the residual sum of squares is minimized. Several variants are suggested in the ways in which the expect­ation is calculated. A parametric (assuming a normal error model) and two non-parametric approaches are described. We also present a method for solving the maximum likelihood equations in the estimation of the regression parameters in the censored regression situation. It is shown that the solutions can be obtained by a recursive algorithm which needs only a least squares routine for optimization. The suggested procesures gain considerably in computational officiency. The Stanford Heart Transplant data is used to illustrate the various methods.     

An asymptotic theory for semiparametric generalized least squares estimation in partially linear regression models

Consider a partially linear regression model with an unknown vector parameter β, an unknown functiong(·), and unknown heteroscedastic error variances. In this paper we develop an asymptotic semiparametric generalized least squares estimation theory under some weak moment conditions. These moment conditions are satisfied by many of the error distributions encountered in practice, and our theory does not require the number of replications to go to infinity.     

Least squares and generalized least squares in models with orthogonal block structure

Besides the basic model, Kronecker products of rotated models are used to isolate the variance components as parameters of a linear model. A characterization of BLUE given by Zmy?lony (1980) is applied to the different models. Generalized least squares are used to complete the estimation.     

On the efficiency of two-stage and three-stage least squares estimators

This paper utilizes the results of Kruskal (1968), Zyskind (1967), and more recently Milliken and Albohali (1984) to derive a simple necessary and sufficient condition for 3SLS to be equivalent to 2SLS. This condition depends upon the inverse of the variance:covariance matrix of the disturbances, and the set of second stage regressors of each structural equation. More importantly, this condition should prove useful for econometric students and provides an easy method for checking sufficiency.     

On the efficiency of two-stage and three-stage least squares estimators

This paper utilizes the results of Kruskal (1968), Zyskind (1967), and more recently Milliken and Albohali (1984) to derive a simple necessary and sufficient condition for 3SLS to be equivalent to 2SLS. This condition depends upon the inverse of the variance:covariance matrix of the disturbances, and the set of second stage regressors of each structural equation. More importantly, this condition should prove useful for econometric students and provides an easy method for checking sufficiency.     

Second-order least squares estimation of censored regression models

This paper proposes the second-order least squares estimation, which is an extension of the ordinary least squares method, for censored regression models where the error term has a general parametric distribution (not necessarily normal). The strong consistency and asymptotic normality of the estimator are derived under fairly general regularity conditions. We also propose a computationally simpler estimator which is consistent and asymptotically normal under the same regularity conditions. Finite sample behavior of the proposed estimators under both correctly and misspecified models are investigated through Monte Carlo simulations. The simulation results show that the proposed estimator using optimal weighting matrix performs very similar to the maximum likelihood estimator, and the estimator with the identity weight is more robust against the misspecification.     

Two stage least squares estimation in structural cointegration models

This paper considers single-equation estimation of simultaneous equation models with integrated processes. The aim of the paper is to investigate asymptotic and small sample properties of some estimators in this framework. We deal with two groups of estimators: such that originally were designated for reduced form estimation and such for simultaneous equation models. In the first group we deal with Least Squares and Fully Modified Least Squares. The second group comprises Two Stage Least Squares and two modifications of it. The asymptotic analysis in section 2 shows that it is true that all estimators are super-consistent in this context but in principle, only the methods of the second group enable valid inference. Section 3 presents the results of a simulation study which is designed for specific problems of simultaneous equation models. This paper was presented at the European Meeting of the Econometric Society in Istanbul, 1996. The author thanks an anonymous referee for helpful suggestions.