Estimating the duration of dynamic effects with temporally aggregated observations

This paper discusses how an Ordinary Least Squares (OLS) estimator can be used to obtain reasonably accurate estimates of the duration of dynamic effects in a Koyck model framework without knowledge of the true level of temporal aggregation of the data. With proper changes in the analytic derivation, the approach can be extended to other dynamic models.     

Some Asymptotic Properties of Ridge Regression in a System of Seemingly Unrelated Regression Equations

A discussion is made of asymptotic properties of an Operational Ordinary Ridge Regression estimator and comparison is made with the Operational Generalized Least Squares estimator. Also, some simulation experiments are carried showing efficiency gains can be made through the use of de Ridge estimator.     

More Efficient Tests Robust to Heteroskedasticity of Unknown Form

In the presence of heteroskedasticity of unknown form, the Ordinary Least Squares parameter estimator becomes inefficient, and its covariance matrix estimator inconsistent. Eicker (1963) and White (1980) were the first to propose a robust consistent covariance matrix estimator, that permits asymptotically correct inference. This estimator is widely used in practice. Cragg (1983) proposed a more efficient estimator, but concluded that tests basd on it are unreliable. Thus, this last estimator has not been used in practice. This article is concerned with finite sample properties of tests robust to heteroskedasticity of unknown form. Our results suggest that reliable and more efficient tests can be obtained with the Cragg estimators in small samples.     

Asymptotic confidence intervals in ridge regression based on the Edgeworth expansion

Ridge Regression techniques have been found useful to reduce mean square errors of parameter estimates when multicollinearity is present. But the usefulness of the method rest not only upon its ability to produce good parameter estimates, with smaller mean squared error than Ordinary Least Squares, but also on having reasonable inferential procedures. The aim of this paper is to develop asymptotic confidence intervals for the model parameters based on Ridge Regression estimates and the Edgeworth expansion. Some simulation experiments are carried out to compare these confidence intervals with those obtained from the application of Ordinary Least Squares. Also, an example will be provided based on the well known data set of Hald.     

A non stochastic ridge regression estimator and comparison with the James-Stein estimator

Abstract

This article presents a non-stochastic version of the Generalized Ridge Regression estimator that arises from a discussion of the properties of a Generalized Ridge Regression estimator whose shrinkage parameters are found to be close to their upper bounds. The resulting estimator takes the form of a shrinkage estimator that is superior to both the Ordinary Least Squares estimator and the James-Stein estimator under certain conditions. A numerical study is provided to investigate the range of signal to noise ratio under which the new estimator dominates the James-Stein estimator with respect to the prediction mean square error.     

Generalized mean squared error properties of regression estimators

Theobald (1974) compares Ordinary Least Squares and Ridge Regression estimators of regression parameters using a generalized mean squared error criterion. This paper presents the generalized mean squared error of a Principal Components Regression estimator and comparisons are made with each of the above estimators. In general the choice of which estimator to use depends on the magnitude and the orientation of the unknown parameter vector.     

Study of partial least squares and ridge regression methods

This article considers both Partial Least Squares (PLS) and Ridge Regression (RR) methods to combat multicollinearity problem. A simulation study has been conducted to compare their performances with respect to Ordinary Least Squares (OLS). With varying degrees of multicollinearity, it is found that both, PLS and RR, estimators produce significant reductions in the Mean Square Error (MSE) and Prediction Mean Square Error (PMSE) over OLS. However, from the simulation study it is evident that the RR performs better when the error variance is large and the PLS estimator achieves its best results when the model includes more variables. However, the advantage of the ridge regression method over PLS is that it can provide the 95% confidence interval for the regression coefficients while PLS cannot.     

Higher order generalisation of first order autoregressive tests

Higher order analogues of currently favoured tests for autocorrelated dicturbances are presented. Comparisons of power, against autoregressive disturbances of a range of orders considered likely in practice, favour a version of a test originally proposed by King (1985). This superiority is particularly marked with regressors for which Ordinary Least Squares is least efficient and thus testing most crucial. Selected significance bounds are tabulated for the twelfth order analogue of this test statistic for use with monthly data.     

To be or not to be valid in testing the significance of the slope in simple quantitative linear models with autocorrelated errors

In this article, the validity of procedures for testing the significance of the slope in quantitative linear models with one explanatory variable and first-order autoregressive [AR(1)] errors is analyzed in a Monte Carlo study conducted in the time domain. Two cases are considered for the regressor: fixed and trended versus random and AR(1). In addition to the classical t -test using the Ordinary Least Squares (OLS) estimator of the slope and its standard error, we consider seven t -tests with n-2\,\hbox{df} built on the Generalized Least Squares (GLS) estimator or an estimated GLS estimator, three variants of the classical t -test with different variances of the OLS estimator, two asymptotic tests built on the Maximum Likelihood (ML) estimator, the F -test for fixed effects based on the Restricted Maximum Likelihood (REML) estimator in the mixed-model approach, two t -tests with n - 2 df based on first differences (FD) and first-difference ratios (FDR), and four modified t -tests using various corrections of the number of degrees of freedom. The FDR t -test, the REML F -test and the modified t -test using Dutilleul's effective sample size are the most valid among the testing procedures that do not assume the complete knowledge of the covariance matrix of the errors. However, modified t -tests are not applicable and the FDR t -test suffers from a lack of power when the regressor is fixed and trended ( i.e. , FDR is the same as FD in this case when observations are equally spaced), whereas the REML algorithm fails to converge at small sample sizes. The classical t -test is valid when the regressor is fixed and trended and autocorrelation among errors is predominantly negative, and when the regressor is random and AR(1), like the errors, and autocorrelation is moderately negative or positive. We discuss the results graphically, in terms of the circularity condition defined in repeated measures ANOVA and of the effective sample size used in correlation analysis with autocorrelated sample data. An example with environmental data is presented.     

Comparison of estimators for heteroskedastic models

Ordinary least squares (OLS) yield inefficient parameter estimates and inconsistent estimates of the covariance matrix in case of heteroskedastic errors. Robinson's adaptive estimator and the Cragg estimator avoid any explicit parameterization of heteroskedasticity, and reduce the danger of misspecification. A small Monte Carlo experiment is performed to compare the behavior of the adaptive estimator with the performance of the Cragg estimator. The Monte Carlo experiment includes simulations of the Generalized Least Squares (GLS) estimator. Indeed, an interesting question is how more sophisticated techniques, like the adaptive estimator, compare with GLS when the latter relies on an incorrect specification of the heteroskedastic process. It turns out that the regression parameters, when estimated adaptively, display small mean squared errors and great efficiency in case of medium or high heteroskedasticity. The covariance matrix, instead, is better estimated by the Cragg estimator or by GLS based on a misspecified error term, since the adaptive estimator overpredicts the standard errors of the regression parameters.     

A Novel Regression Approach: Least Squares Ratio

Regression Analysis (RA) is one of the frequently used tool for forecasting. The Ordinary Least Squares (OLS) Technique is the basic instrument of RA and there are many regression techniques based on OLS. This paper includes a new regression approach, called Least Squares Ratio (LSR), and comparison of OLS and LSR according to mean square errors of estimation of theoretical regression parameters (mse ß) and dependent value (mse y).     

A NOTE ON ESTIMATING LINEAR TREND IN A REGRESSION MODEL WITH SERIALLY CORRELATED ERROR COMPONENTS

ABSTRACT

We consider a linear trend regression model when the disturbances follow a serially correlated one-way error component model. In this model, we investigate the performance of the Ordinary Least Squares Esitmator (OLSE), First Difference Estimator (FDE), Generalized Least Squares Estimator (GLSE) and the Cochrane-Orcutt-Transformation Estimator (COTE) of slope coefficient in terms of efficiency. The main findings are as follows: (1) when the autocorrelation is close to unity, then the FDE is approximately the GLSE; (2) the OLSE is better than the COTE; and (3) when the value of the autocorrelation is kept constant and T → ∞, the OLSE, COTE and GLSE are asymptotically equivalent whereas the FDE is worse than the other estimators in terms of efficiency.     

Towards Uniformly Efficient Trend Estimation Under Weak/Strong Correlation and Non‐stationary Volatility

In this paper, we consider the deterministic trend model where the error process is allowed to be weakly or strongly correlated and subject to non‐stationary volatility. Extant estimators of the trend coefficient are analysed. We find that under heteroskedasticity, the Cochrane–Orcutt‐type estimator (with some initial condition) could be less efficient than Ordinary Least Squares (OLS) when the process is highly persistent, whereas it is asymptotically equivalent to OLS when the process is less persistent. An efficient non‐parametrically weighted Cochrane–Orcutt‐type estimator is then proposed. The efficiency is uniform over weak or strong serial correlation and non‐stationary volatility of unknown form. The feasible estimator relies on non‐parametric estimation of the volatility function, and the asymptotic theory is provided. We use the data‐dependent smoothing bandwidth that can automatically adjust for the strength of non‐stationarity in volatilities. The implementation does not require pretesting persistence of the process or specification of non‐stationary volatility. Finite‐sample evaluation via simulations and an empirical application demonstrates the good performance of proposed estimators.     

OLS与ML：回归模型两种参数估计方法的比较研究

最小二乘法(OLS)和最大似然法(ML)是回归模型参数估计的两种最重要的方法。 但二者有着明显的差别,本文就二者之间的有关差别进行比较。     

Spurious Instrumental Variables

Spurious regression phenomenon has been recognized for a wide range of Data Generating Processes: driftless unit roots, unit roots with drift, long memory, trend and broken-trend stationarity, etc. The usual framework is Ordinary Least Squares. We show that the spurious phenomenon also occurs in Instrumental Variables estimation when using non stationary variables, whether the non stationarity component is stochastic or deterministic. Finite sample evidence supports the asymptotic results.     

A comparison of six methods of analysing row-column designs with inter-block information

Six methods of obtaining estimates of treatment effects in a row-column design are considered. Five methods use estimates of inter-row and inter-column variation, and the remaining method is Ordinary Least Squares. Using simulation, these methods are examined to see which are most appropriate for minimising the sum of the squared differences between the estimates of the elementary treatment contrasts and their true values. Recommendations are made of which methods to use.     

Errors in variables: consistent adjusted least squares (cals) estimation

A single equation errors-in-variables model is considered. Exact restrictions on the parameters in the model are assumed to be available such that the model is just-identified. A Consistent Adjusted Least Squares (CALS) estimator for this model is proposed and its asymptotic distribution is given. Special cases are given as illustrations. CALS is identical to the Method of Moments (MM), and to Maximum Likelihood (ML) under the structural interpretation. Under the functional interpretation it is identical to ML in cases where the latter method is consistent.     

Efficiency analysis of ten estimation procedures for quantitative linear models with autocorrelated errors

Many estimation procedures for quantitative linear models with autocorrelated errors have been proposed in the literature. A number of these procedures have been compared in various ways for different sample sizes and autocorrelation parameters values and for structured or random explanatory vaiables. In this paper, we revisit three situations that were considered to some extent in previous studies, by comparing ten estimation procedures: Ordinary Least Squares (OLS), Generalized Least Squares (GLS), estimated Generalized Least Squares (six procedures), Maximum Likelihood (ML), and First Differences (FD). The six estimated GLS procedures and the ML procedure differ in the way the error autocovariance matrix is estimated. The three situations can be defined as follows: Case 1, the explanatory variable x in the simple linear regression is fixed; Case 2,x is purely random; and Case 3x is first-order autoregressive. Following a theoretical presentation, the ten estimation procedures are compared in a Monte Carlo study conducted in the time domain, where the errors are first-order autoregressive in Cases 1-3. The measure of comparison for the estimation procedures is their efficiency relative to OLS. It is evaluated as a function of the time series length and the magnitude and sign of the error autocorrelation parameter. Overall, knowledge of the model of the time series process generating the errors enhances efficiency in estimated GLS. Differences in the efficiency of estimation procedures between Case 1 and Cases 2 and 3 as well as differences in efficiency among procedures in a given situation are observed and discussed.     

Nonlinear unbiased estimation in the linear regression model with nonnormal disturbances

In the application of the linear regression model there continues to be wide-spread use of the Least Squares Estimator (LSE) due to its theoretical optimality. For example, it is well known that the LSE is the best unbiased estimator under normality while it remains best linear unbiased estimator (BLUE) when the normality assumption is dropped. In this paper we extend an approach given in Knautz (1993) that allows improvement of the LSE in the context of nonnormal and nonsymmetric error distributions. It will be shown that there exist linear plus quadratic (LPQ) estimators, consisting of linear and quadratic terms in the dependent variable, which dominate the LS estimator, depending on second, third and fourth moments of the error distribution. A simulation study illustrates that this remains true if the moments have to be estimated from the data. Computation of confidence intervals using bootstrap methods reveal significant improvement compared with inference based on the LS especially for nonsymmetric distributions of the error term.