Forecasting with serially correlated regression models

In this article we investigate the asymptotic and finite-sample properties of predictors of regression models with autocorrelated errors. We prove new theorems associated with the predictive efficiency of generalized least squares (GLS) and incorrectly structured GLS predictors. We also establish the form associated with their predictive mean squared errors as well as the magnitude of these errors relative to each other and to those generated from the ordinary least squares (OLS) predictor. A large simulation study is used to evaluate the finite-sample performance of forecasts generated from models using different corrections for the serial correlation.     

Comparison of estimators for heteroskedastic models

Ordinary least squares (OLS) yield inefficient parameter estimates and inconsistent estimates of the covariance matrix in case of heteroskedastic errors. Robinson's adaptive estimator and the Cragg estimator avoid any explicit parameterization of heteroskedasticity, and reduce the danger of misspecification. A small Monte Carlo experiment is performed to compare the behavior of the adaptive estimator with the performance of the Cragg estimator. The Monte Carlo experiment includes simulations of the Generalized Least Squares (GLS) estimator. Indeed, an interesting question is how more sophisticated techniques, like the adaptive estimator, compare with GLS when the latter relies on an incorrect specification of the heteroskedastic process. It turns out that the regression parameters, when estimated adaptively, display small mean squared errors and great efficiency in case of medium or high heteroskedasticity. The covariance matrix, instead, is better estimated by the Cragg estimator or by GLS based on a misspecified error term, since the adaptive estimator overpredicts the standard errors of the regression parameters.     

Efficiency analysis of ten estimation procedures for quantitative linear models with autocorrelated errors

Many estimation procedures for quantitative linear models with autocorrelated errors have been proposed in the literature. A number of these procedures have been compared in various ways for different sample sizes and autocorrelation parameters values and for structured or random explanatory vaiables. In this paper, we revisit three situations that were considered to some extent in previous studies, by comparing ten estimation procedures: Ordinary Least Squares (OLS), Generalized Least Squares (GLS), estimated Generalized Least Squares (six procedures), Maximum Likelihood (ML), and First Differences (FD). The six estimated GLS procedures and the ML procedure differ in the way the error autocovariance matrix is estimated. The three situations can be defined as follows: Case 1, the explanatory variable x in the simple linear regression is fixed; Case 2,x is purely random; and Case 3x is first-order autoregressive. Following a theoretical presentation, the ten estimation procedures are compared in a Monte Carlo study conducted in the time domain, where the errors are first-order autoregressive in Cases 1-3. The measure of comparison for the estimation procedures is their efficiency relative to OLS. It is evaluated as a function of the time series length and the magnitude and sign of the error autocorrelation parameter. Overall, knowledge of the model of the time series process generating the errors enhances efficiency in estimated GLS. Differences in the efficiency of estimation procedures between Case 1 and Cases 2 and 3 as well as differences in efficiency among procedures in a given situation are observed and discussed.     

Ridge estimation in regression problems with autocorrelated errors: A monte carlo study

This paper presents the results of a Monte Carlo study of OLS and GLS based adaptive ridge estimators for regression problems in which the independent variables are collinear and the errors are autocorrelated. It studies the effects of degree of collinearity, magnitude of error variance, orientation of the parameter vector and serial correlation of the independent variables on the mean squared error performance of these estimators. Results suggest that such estimators produce greatly improved performance in favorable portions of the parameter space. The GLS based methods are best when the independent variables are also serially correlated.     

Nonparametric regression for dependent data in the errors-in-variables problem

We consider the nonparametric estimation of the regression functions for dependent data. Suppose that the covariates are observed with additive errors in the data and we employ nonparametric deconvolution kernel techniques to estimate the regression functions in this paper. We investigate how the strength of time dependence affects the asymptotic properties of the local constant and linear estimators. We treat both short-range dependent and long-range dependent linear processes in a unified way and demonstrate that the long-range dependence (LRD) of the covariates affects the asymptotic properties of the nonparametric estimators as well as the LRD of regression errors does.     

An improved meta-analysis for analyzing cylindrical-type time series data with applications to forecasting problem in environmental study

In this paper, we propose an improved generalized least square (GLS) meta-analysis in a linear-circular regression, and show its utility in the analysis of a certain environmental issue. The existing GLS meta-analysis proposed in Becker and Wu has a serious flaw since information about the covariance among coefficients across studies is not utilized. In our proposed meta-analysis, we take the correlations between adjacent studies into account, and improve the existing GLS meta-analysis. We provide numerical examples to compare the proposed method with several other existing methods by using Akaike's Information Criterion, Bayesian Information Criterion and mean square prediction errors with applications to forecasting problem in Environmental study.     

Artificial neural networks for some macroeconomic series: A first report

There is a tendency for the true variability of feasible GLS estimators to be understated by asymptotic standard errors. For estimation of SUR models, this tendency becomes more severe in large equation systems when estimation of the error covariance matrix, C, becomes problematic. We explore a number of potential solutions involving the use of improved estimators for the disturbance covariance matrix and bootstrapping. In particular, Ullah and Racine (1992) have recently introduced a new class of estimators for SUR models that use nonparametric kernel density estimation techniques. The proposed estimators have the same structure as the feasible GLS estimator of Zellner (1962) differing only in the choice of estimator for C. Ullah and Racine (1992) prove that their nonparametric density estimator of C can be expressed as Zellner's original estimator plus a positive definite matrix that depends on the smoothing parameter chosen for the density estimation. It is this structure of the estimator that most interests us as it has the potential to be especially useful in large equation systems.

Atkinson and Wilson (1992) investigated the bias in the conventional and bootstrap estimators of coefficient standard errors in SUR models. They demonstrated that under certain conditions the former were superior, but they caution that neither estimator uniformly dominated and hence bootstrapping provides little improvement in the estimation of standard errors for the regression coefficients. Rilstone and Veal1 (1996) argue that an important qualification needs to be made to this somewhat negative conclusion. They demonstrated that bootstrapping can result in improvements in inferences if the procedures are applied to the t-ratios rather than to the standard errors. These issues are explored for the case of large equation systems and when bootstrapping is combined with improved covariance estimation.     

The frisch-waugh theorem and generalized least squares

We discuss the standard linear regression model with nonspherical disturbances, where some regressors are annihilated by considering only the residuals from an auxiliary regression, and where, analogous to the Frisch-Waugh procedure, the original GLS procedure is applied to the transformed data. We call this procedure pseudo-GLS and give conditions for pseudo-GL5 to be equal to genuine GLS. We also show via examples that these conditions are often violated in empirical applications, and that the Frisch-Waugh Theorem still “works” with nonspherical disturbances if efficient estimation is applied to both the original and the transformed data.     

The frisch-waugh theorem and generalized least squares

We discuss the standard linear regression model with nonspherical disturbances, where some regressors are annihilated by considering only the residuals from an auxiliary regression, and where, analogous to the Frisch-Waugh procedure, the original GLS procedure is applied to the transformed data. We call this procedure pseudo-GLS and give conditions for pseudo-GL5 to be equal to genuine GLS. We also show via examples that these conditions are often violated in empirical applications, and that the Frisch-Waugh Theorem still “works” with nonspherical disturbances if efficient estimation is applied to both the original and the transformed data.     

The asymptotic behaviour of a class of L.-estimators under long-range dependence

This paper obtains asymptotic representations of a class of L-estimators in a linear regression model when the errors are a function of long-range-dependent Gaussian random variables. These representations are then used to address some of the efficiency robustness properties of L-estimators compared to the least-squares estimator. It is observed that under the Gaussian error distribution, each member of the class has the same asymptotic efficiency as that of the least-squares estimator. The results are obtained as a consequence of the asymptotic uniform linearity of some weighted empirical processes based on long-range-dependent random variables.     

Generalized least squares with misspecified serial correlation structures

Summary. The regression literature contains hundreds of studies on serially correlated disturbances. Most of these studies assume that the structure of the error covariance matrix Ω is known or can be estimated consistently from data. Surprisingly, few studies investigate the properties of estimated generalized least squares (GLS) procedures when the structure of Ω is incorrectly identified and the parameters are inefficiently estimated. We compare the finite sample efficiencies of ordinary least squares (OLS), GLS and incorrect GLS (IGLS) estimators. We also prove new theorems establishing theoretical efficiency bounds for IGLS relative to GLS and OLS. Results from an exhaustive simulation study are used to evaluate the finite sample performance and to demonstrate the robustness of IGLS estimates vis-à-vis OLS and GLS estimates constructed for models with known and estimated (but correctly identified) Ω. Some of our conclusions for finite samples differ from established asymptotic results.     

The error components regression model: conditional relative efficiency comparisons

This paper considers a simple linear regression with two-way error component disturbances and derives the conditional relative efficiency ofany feasible GLS estimator with respect to OLS, true GLS, orany other feasible GLS estimator, conditional on the estimated variance components. This is done at two crucial choices of the x variable. The first choice is where OLS is least efficient with respect to GLS and the second choice is where an arbitrary feasible GLS estimator is least efficient with respect to GLS. Our findings indicate that a better guess of a certain ‘variance components ratio’ leads to better estimates of the regression coefficients.     

Two-Step Residual-Based Estimation of Error Variances for Generalized Least Squares in Split-Plot Experiments

In split-plot experiments, estimation of unknown parameters by generalized least squares (GLS), as opposed to ordinary least squares (OLS), is required, owing to the existence of whole- and subplot errors. However, estimating the error variances is often necessary for GLS. Restricted maximum likelihood (REML) is an established method for estimating the error variances, and its benefits have been highlighted in many previous studies. This article proposes a new two-step residual-based approach for estimating error variances. Results of numerical simulations indicate that the proposed method performs sufficiently well to be considered as a suitable alternative to REML.     

OPTIMAL IV ESTIMATION OF SYSTEMS WITH STOCHASTIC REGRESSORS AND VAR DISTURBANCES WITH APPLICATIONS TO DYNAMIC SYSTEMS

This paper considers the general problem of Feasible Generalized Least Squares Instrumental Variables (FGLS IV) estimation using optimal instruments. First we summarize the sufficient conditions for the FGLS IV estimator to be asymptotically equivalent to an optimal GLS IV estimator. Then we specialize to stationary dynamic systems with stationary VAR errors, and use the sufficient conditions to derive new moment conditions for these models. These moment conditions produce useful IVs from the lagged endogenous variables, despite the correlation between errors and endogenous variables. This use of the information contained in the lagged endogenous variables expands the class of IV estimators under consideration and thereby potentially improves both asymptotic and small-sample efficiency of the optimal IV estimator in the class. Some Monte Carlo experiments compare the new methods with those of Hatanaka (1976). For the DGP used in the Monte Carlo experiments, asymptotic efficiency is strictly improved by the new IVs, and experimental small-sample efficiency is improved as well.     

OPTIMAL IV ESTIMATION OF SYSTEMS WITH STOCHASTIC REGRESSORS AND VAR DISTURBANCES WITH APPLICATIONS TO DYNAMIC SYSTEMS

This paper considers the general problem of Feasible Generalized Least Squares Instrumental Variables (FGLS IV) estimation using optimal instruments. First we summarize the sufficient conditions for the FGLS IV estimator to be asymptotically equivalent to an optimal GLS IV estimator. Then we specialize to stationary dynamic systems with stationary VAR errors, and use the sufficient conditions to derive new moment conditions for these models. These moment conditions produce useful IVs from the lagged endogenous variables, despite the correlation between errors and endogenous variables. This use of the information contained in the lagged endogenous variables expands the class of IV estimators under consideration and thereby potentially improves both asymptotic and small-sample efficiency of the optimal IV estimator in the class. Some Monte Carlo experiments compare the new methods with those of Hatanaka (1976). For the DGP used in the Monte Carlo experiments, asymptotic efficiency is strictly improved by the new IVs, and experimental small-sample efficiency is improved as well.     

Estimation of the seemingly unrelated regression model when the error covariance matrix is singular

A two-step generalized least-squares (GLS) estimator proposed by Zellner for seemingly unrelated regression (SUR) models is implementable when the estimated covariance matrix of the errors in the SUR system is non-singular. Violating the premise of non-singularity is a common problem among many applications in economics, business and management. We present methods of resolving this problem and propose an efficient procedure. The simulation study shows that the estimator of Haff performs better for small-sized observations, whereas the estimator of Ullah and Racine performs better for larger sized observations. Furthermore, the Ullah-Racine estimate is simple to calculate and easy to use. The empirical analysis involves the study of the diffusion processes of videocassette recorders across different geographic regions in the US, which exhibits a singular covariance matrix. The empirical results show that the procedures efficiently deal with the problem and provide plausible estimation results.     

Adaptive estimation of vector autoregressive models with time-varying variance: Application to testing linear causality in mean

Linear vector autoregressive (VAR) models where the innovations could be unconditionally heteroscedastic are considered. The volatility structure is deterministic and quite general, including breaks or trending variances as special cases. In this framework we propose ordinary least squares (OLS), generalized least squares (GLS) and adaptive least squares (ALS) procedures. The GLS estimator requires the knowledge of the time-varying variance structure while in the ALS approach the unknown variance is estimated by kernel smoothing with the outer product of the OLS residual vectors. Different bandwidths for the different cells of the time-varying variance matrix are also allowed. We derive the asymptotic distribution of the proposed estimators for the VAR model coefficients and compare their properties. In particular we show that the ALS estimator is asymptotically equivalent to the infeasible GLS estimator. This asymptotic equivalence is obtained uniformly with respect to the bandwidth(s) in a given range and hence justifies data-driven bandwidth rules. Using these results we build Wald tests for the linear Granger causality in mean which are adapted to VAR processes driven by errors with a nonstationary volatility. It is also shown that the commonly used standard Wald test for the linear Granger causality in mean is potentially unreliable in our framework (incorrect level and lower asymptotic power). Monte Carlo experiments illustrate the use of the different estimation approaches for the analysis of VAR models with time-varying variance innovations.     

Generalized Least Squares Estimation in Contingency Tables Analysis: Asymptotic Properties and Applications

Different sorts of bilinear models (models with bilinear interaction terms) are currently used when analyzing contingency tables: association models, correlation models... All these can be included in a general family of bilinear models: power models. In this framework, Maximum Likelihood (ML) estimation is not always possible, as explained in an introductory example. Thus, Generalized Least Squares (GLS) estimation is sometimes needed in order to estimate parameters. A subclass of power models is then considered in this paper: separable reduced-rank (SRR) models. They allow an optimal choice of weights for GLS estimation and simplifications in asymptotic studies concerning GLS estimators. Power 2 models belong to the subclass of SRR models and the asymptotic properties of GLS estimators are established. Similar results are also established for association models which are not SRR models. However, these results are more difficult to prove. Finally, 2 examples are considered to illustrate our results.     

Estimated Generalized Least Squares for a Heteroscedastic Regression Model

This paper considers the general linear regression model yc = X₁β+u_t under the heteroscedastic structure E(u_t) = 0, E(u²) =σ²- (X_tβ)², E(u_t u_s) = 0, tæs, t, s= 1, T. It is shown that any estimated GLS estimator for β is asymptotically equivalent to the GLS estimator under some regularity conditions. A three-step GLS estimator, which calls upon the assumption E(u_t²) =s?²(X,β)² for the estimation of the disturbance covariance matrix, is considered.     

A note on fourth moment-based direction dependence measures when regression errors are non normal

Measures of direction dependence enable researchers to determine the directionality of linear effects in bivariate data. Existing fourth moment-based approaches assume that regression errors are at least mesokurtic. Direction dependence measures based on the co-kurtosis of variables are proposed that relax this assumption. Simulations suggest that co-kurtosis-based measures perform equally well as existing kurtosis-based methods when distributional assumptions of the latter are fulfilled. However, kurtosis-based approaches are sensitive to platy- or leptokurtic errors, while co-kurtosis-based measures protect Type I error and power rates. Data requirements necessary for causal inference and recommendations for selecting proper direction dependence measures are discussed.