The frisch-waugh theorem and generalized least squares

We discuss the standard linear regression model with nonspherical disturbances, where some regressors are annihilated by considering only the residuals from an auxiliary regression, and where, analogous to the Frisch-Waugh procedure, the original GLS procedure is applied to the transformed data. We call this procedure pseudo-GLS and give conditions for pseudo-GL5 to be equal to genuine GLS. We also show via examples that these conditions are often violated in empirical applications, and that the Frisch-Waugh Theorem still “works” with nonspherical disturbances if efficient estimation is applied to both the original and the transformed data.     

The error components regression model: conditional relative efficiency comparisons

This paper considers a simple linear regression with two-way error component disturbances and derives the conditional relative efficiency ofany feasible GLS estimator with respect to OLS, true GLS, orany other feasible GLS estimator, conditional on the estimated variance components. This is done at two crucial choices of the x variable. The first choice is where OLS is least efficient with respect to GLS and the second choice is where an arbitrary feasible GLS estimator is least efficient with respect to GLS. Our findings indicate that a better guess of a certain ‘variance components ratio’ leads to better estimates of the regression coefficients.     

Generalized least squares with misspecified serial correlation structures

Summary. The regression literature contains hundreds of studies on serially correlated disturbances. Most of these studies assume that the structure of the error covariance matrix Ω is known or can be estimated consistently from data. Surprisingly, few studies investigate the properties of estimated generalized least squares (GLS) procedures when the structure of Ω is incorrectly identified and the parameters are inefficiently estimated. We compare the finite sample efficiencies of ordinary least squares (OLS), GLS and incorrect GLS (IGLS) estimators. We also prove new theorems establishing theoretical efficiency bounds for IGLS relative to GLS and OLS. Results from an exhaustive simulation study are used to evaluate the finite sample performance and to demonstrate the robustness of IGLS estimates vis-à-vis OLS and GLS estimates constructed for models with known and estimated (but correctly identified) Ω. Some of our conclusions for finite samples differ from established asymptotic results.     

Semiparametric Sieve-Type Generalized Least Squares Inference

This article considers the problem of statistical inference in linear regression models with dependent errors. A sieve-type generalized least squares (GLS) procedure is proposed based on an autoregressive approximation to the generating mechanism of the errors. The asymptotic properties of the sieve-type GLS estimator are established under general conditions, including mixingale-type conditions as well as conditions which allow for long-range dependence in the stochastic regressors and/or the errors. A Monte Carlo study examines the finite-sample properties of the method for testing regression hypotheses.     

STAR模型中的递归退势单位根检验研究

在STAR模型框架下,考虑时间序列具有线性确定性趋势成分,本文建立了一个递归退势单位根检验统计量,推导了其渐近分布;并在考虑初始条件情形下,对递归退势、OLS和GLS退势单位根检验统计量的有限样本性质进行了细致的比较研究。若忽略初始条件的影响,GLS退势和递归退势单位根检验统计量的检验势都显著高于OLS退势。随着初始条件的增大,GLS退势单位根检验统计量的检验势下降得比较厉害,递归退势单位根检验统计量的检验势较为稳定,且在样本量较大情形下更具优势。     

A transformation for estimating the trinomial logit model with grouped data

For logit models where the outcome variables are the proportions of individuals falling into each of three categories, this paper develops a data transformation through which GLS estimates can be obtained by running OLS on the transformed data.     

Alternative transformations to eliminate fixed effects

In a panel data model with fixed individual effects, a number of alternative transformations are available to eliminate these effects such that the slope parameters can be estimated from ordinary least squares on transformed data. In this note we show that each transformation leads to algebraically the same estimator if the transformed data are used efficiently (i.e. if GLS is applied). If OLS is used, however, differences may occur and the routinely computed variances, even after degrees of freedom correction, are incorrect. In addition, it may matter whether “redundant” observations are used or not.     

Ridge regression estimators in the linear regression models with non-spherical errors

The present paper considers a family of ordinary ridge regression estimators in the linear regression model when the disturbances covariance matrix depends upon a few unknown parameters. An asymptotic expansion for the distribution of the ridge regression estimator is developed and under the quadratic loss function its asymptotic risk is compared with that of the feasible GLS estimator.     

A Balanced System of U.S. Industry Accounts and Distribution of the Aggregate Statistical Discrepancy by Industry

This article describes and illustrates a generalized least squares (GLS) method that systematically incorporates all available information on the reliability of initial data in the reconciliation of a large disaggregated system of national accounts. The GLS method is applied to reconciling the 1997 U.S. Input-Output and Gross Domestic Product (GDP)-by-industry accounts with benchmarked GDP estimated from expenditures. The GLS procedure produced a balanced system of industry accounts and distributed the aggregate statistical discrepancy by industry according to the estimated relative reliabilities of initial estimates. The study demonstrates the empirical feasibility and computational efficiency of the GLS method for large accounts reconciliation.     

A generalized Bayesian nonlinear mixed‐effects regression model for zero‐inflated longitudinal count data in tuberculosis trials

In this paper, we investigate Bayesian generalized nonlinear mixed‐effects (NLME) regression models for zero‐inflated longitudinal count data. The methodology is motivated by and applied to colony forming unit (CFU) counts in extended bactericidal activity tuberculosis (TB) trials. Furthermore, for model comparisons, we present a generalized method for calculating the marginal likelihoods required to determine Bayes factors. A simulation study shows that the proposed zero‐inflated negative binomial regression model has good accuracy, precision, and credibility interval coverage. In contrast, conventional normal NLME regression models applied to log‐transformed count data, which handle zero counts as left censored values, may yield credibility intervals that undercover the true bactericidal activity of anti‐TB drugs. We therefore recommend that zero‐inflated NLME regression models should be fitted to CFU count on the original scale, as an alternative to conventional normal NLME regression models on the logarithmic scale.     

Estimation of linear models for field experiments

General mixed linear models for experiments conducted over a series of sltes and/or years are described. The ordinary least squares (OLS) estlmator is simple to compute, but is not the best unbiased estimator. Also, the usuaL formula for the varlance of the OLS estimator is not correct and seriously underestimates the true variance. The best linear unbiased estimator is the generalized least squares (GLS) estimator. However, t requires an inversion of the variance-covariance matrix V, whlch is usually of large dimension. Also, in practice, V is unknown.

We presented an estlmator [Vcirc] of the matrix V using the estimators of variance components [for sites, blocks (sites), etc.]. We also presented a simple transformation of the data, such that an ordinary least squares regression of the transformed data gives the estimated generalized least squares (EGLS) estimator. The standard errors obtained from the transformed regression serve as asymptotic standard errors of the EGLS estimators. We also established that the EGLS estlmator is unbiased.

An example of fitting a linear model to data for 18 sites (environments) located in Brazil is given. One of the site variables (soil test phosphorus) was measured by plot rather than by site and this established the need for a covariance model such as the one used rather than the usual analysis of variance model. It is for this variable that the resulting parameter estimates did not correspond well between the OLS and EGLS estimators. Regression statistics and the analysis of variance for the example are presented and summarized.     

Transformations for Smooth Regression Models with Multiplicative Errors

We consider whether one should transform to estimate nonparametrically a regression curve sampled from data with a constant coefficient of variation, i.e. with multiplicative errors. Kernel-based smoothing methods are used to provide curve estimates from the data both in the original units and after transformation. Comparisons are based on the mean-squared error (MSE) or mean integrated squared error (MISE), calculated in the original units. Even when the data are generated by the simplest multiplicative error model, the asymptotically optimal MSE (or MISE) is surprisingly not always obtained by smoothing transformed data, but in many cases by directly smoothing the original data. Which method is optimal depends on both the regression curve and the distribution of the errors. Data-based procedures which could be useful in choosing between transforming and not transforming a particular data set are discussed. The results are illustrated on simulated and real data.     

Artificial neural networks for some macroeconomic series: A first report

There is a tendency for the true variability of feasible GLS estimators to be understated by asymptotic standard errors. For estimation of SUR models, this tendency becomes more severe in large equation systems when estimation of the error covariance matrix, C, becomes problematic. We explore a number of potential solutions involving the use of improved estimators for the disturbance covariance matrix and bootstrapping. In particular, Ullah and Racine (1992) have recently introduced a new class of estimators for SUR models that use nonparametric kernel density estimation techniques. The proposed estimators have the same structure as the feasible GLS estimator of Zellner (1962) differing only in the choice of estimator for C. Ullah and Racine (1992) prove that their nonparametric density estimator of C can be expressed as Zellner's original estimator plus a positive definite matrix that depends on the smoothing parameter chosen for the density estimation. It is this structure of the estimator that most interests us as it has the potential to be especially useful in large equation systems.

Atkinson and Wilson (1992) investigated the bias in the conventional and bootstrap estimators of coefficient standard errors in SUR models. They demonstrated that under certain conditions the former were superior, but they caution that neither estimator uniformly dominated and hence bootstrapping provides little improvement in the estimation of standard errors for the regression coefficients. Rilstone and Veal1 (1996) argue that an important qualification needs to be made to this somewhat negative conclusion. They demonstrated that bootstrapping can result in improvements in inferences if the procedures are applied to the t-ratios rather than to the standard errors. These issues are explored for the case of large equation systems and when bootstrapping is combined with improved covariance estimation.     

Adaptive Elastic Net GMM Estimation With Many Invalid Moment Conditions: Simultaneous Model and Moment Selection

This article develops the adaptive elastic net generalized method of moments (GMM) estimator in large-dimensional models with potentially (locally) invalid moment conditions, where both the number of structural parameters and the number of moment conditions may increase with the sample size. The basic idea is to conduct the standard GMM estimation combined with two penalty terms: the adaptively weighted lasso shrinkage and the quadratic regularization. It is a one-step procedure of valid moment condition selection, nonzero structural parameter selection (i.e., model selection), and consistent estimation of the nonzero parameters. The procedure achieves the standard GMM efficiency bound as if we know the valid moment conditions ex ante, for which the quadratic regularization is important. We also study the tuning parameter choice, with which we show that selection consistency still holds without assuming Gaussianity. We apply the new estimation procedure to dynamic panel data models, where both the time and cross-section dimensions are large. The new estimator is robust to possible serial correlations in the regression error terms.     

Relative efficiency of first difference estimator in panel data regression with serially correlated error components

We consider the pooled cross-sectional and time series regression model when the disturbances follow a serially correlated one-way error components. In this context we discovered that the first difference estimator for the regression coefficients is equivalent to the generalized least squares estimator irrespective of the particular form of the regressor matrix when the disturbances are generated by a first order autoregressive process where the autocorrelation is close to unity.     

The additive hazards model with high-dimensional regressors

This paper considers estimation and prediction in the Aalen additive hazards model in the case where the covariate vector is high-dimensional such as gene expression measurements. Some form of dimension reduction of the covariate space is needed to obtain useful statistical analyses. We study the partial least squares regression method. It turns out that it is naturally adapted to this setting via the so-called Krylov sequence. The resulting PLS estimator is shown to be consistent provided that the number of terms included is taken to be equal to the number of relevant components in the regression model. A standard PLS algorithm can also be constructed, but it turns out that the resulting predictor can only be related to the original covariates via time-dependent coefficients. The methods are applied to a breast cancer data set with gene expression recordings and to the well known primary biliary cirrhosis clinical data.     

Efficiency analysis of ten estimation procedures for quantitative linear models with autocorrelated errors

Many estimation procedures for quantitative linear models with autocorrelated errors have been proposed in the literature. A number of these procedures have been compared in various ways for different sample sizes and autocorrelation parameters values and for structured or random explanatory vaiables. In this paper, we revisit three situations that were considered to some extent in previous studies, by comparing ten estimation procedures: Ordinary Least Squares (OLS), Generalized Least Squares (GLS), estimated Generalized Least Squares (six procedures), Maximum Likelihood (ML), and First Differences (FD). The six estimated GLS procedures and the ML procedure differ in the way the error autocovariance matrix is estimated. The three situations can be defined as follows: Case 1, the explanatory variable x in the simple linear regression is fixed; Case 2,x is purely random; and Case 3x is first-order autoregressive. Following a theoretical presentation, the ten estimation procedures are compared in a Monte Carlo study conducted in the time domain, where the errors are first-order autoregressive in Cases 1-3. The measure of comparison for the estimation procedures is their efficiency relative to OLS. It is evaluated as a function of the time series length and the magnitude and sign of the error autocorrelation parameter. Overall, knowledge of the model of the time series process generating the errors enhances efficiency in estimated GLS. Differences in the efficiency of estimation procedures between Case 1 and Cases 2 and 3 as well as differences in efficiency among procedures in a given situation are observed and discussed.     

Reducing errors-in-variables bias in linear regression using compact genetic algorithms

A new technique is devised to mitigate the errors-in-variables bias in linear regression. The procedure mimics a 2-stage least squares procedure where an auxiliary regression which generates a better behaved predictor variable is derived. The generated variable is then used as a substitute for the error-prone variable in the first-stage model. The performance of the algorithm is tested by simulation and regression analyses. Simulations suggest the algorithm efficiently captures the additive error term used to contaminate the artificial variables. Regressions provide further credit to the simulations as they clearly show that the compact genetic algorithm-based estimate of the true but unobserved regressor yields considerably better results. These conclusions are robust across different sample sizes and different variance structures imposed on both the measurement error and regression disturbances.     

Estimation of a linear model in transformed variables under microaggregation by individual ranking

We consider the least squares estimation of a linear regression model in transformed variables from a data set that has been microaggregated by means of the individual ranking method. It is shown that the least squares estimators are consistent even in the case where variable transformations are carried out after microaggregation. Applying individual ranking techniques to a data set thus guarantees the analytical validity of the microaggregated data for a wide class of statistical models.     

Local comparison of empirical distributions via nonparametric regression

Given two independent samples of size n and m drawn from univariate distributions with unknown densities f and g, respectively, we are interested in identifying subintervals where the two empirical densities deviate significantly from each other. The solution is built by turning the nonparametric density comparison problem into a comparison of two regression curves. Each regression curve is created by binning the original observations into many small size bins, followed by a suitable form of root transformation to the binned data counts. Turned as a regression comparison problem, several nonparametric regression procedures for detection of sparse signals can be applied. Both multiple testing and model selection methods are explored. Furthermore, an approach for estimating larger connected regions where the two empirical densities are significantly different is also derived, based on a scale-space representation. The proposed methods are applied on simulated examples as well as real-life data from biology.