Variance Estimation in Spatial Regression Using a Non-parametric Semivariogram Based on Residuals

Abstract.  The empirical semivariogram of residuals from a regression model with stationary errors may be used to estimate the covariance structure of the underlying process. For prediction (kriging) the bias of the semivariogram estimate induced by using residuals instead of errors has only a minor effect because the bias is small for small lags. However, for estimating the variance of estimated regression coefficients and of predictions, the bias due to using residuals can be quite substantial. Thus we propose a method for reducing this bias. The adjusted empirical semivariogram is then isotonized and made conditionally negative-definite and used to estimate the variance of estimated regression coefficients in a general estimating equations setup. Simulation results for least squares and robust regression show that the proposed method works well in linear models with stationary correlated errors.     

Spurious Regressions with Time-Series Data: Further Asymptotic Results

A “spurious regression” is one in which the time-series variables are non stationary and independent. It is well known that in this context the OLS parameter estimates and the R ² converge to functionals of Brownian motions, the “t-ratios” diverge in distribution, and the Durbin–Watson statistic converges in probability to zero. We derive corresponding results for some common tests for the normality and homoskedasticity of the errors in a spurious regression.     

Small sample behavior of a robust heteroskedasticity consistent covariance matrix estimator

In heteroskedastic regression models, the least squares (OLS) covariance matrix estimator is inconsistent and inference is not reliable. To deal with inconsistency one can estimate the regression coefficients by OLS, and then implement a heteroskedasticity consistent covariance matrix (HCCM) estimator. Unfortunately the HCCM estimator is biased. The bias is reduced by implementing a robust regression, and by using the robust residuals to compute the HCCM estimator (RHCCM). A Monte-Carlo study analyzes the behavior of RHCCM and of other HCCM estimators, in the presence of systematic and random heteroskedasticity, and of outliers in the explanatory variables.     

A comparison of linear regression techniques in method comparison studies

In this study, the performances of linear regression techniques, which are especially used in clinical chemistry in method comparison studies, are compared via the Monte-Carlo simulation. The regression techniques that take the measurement errors of both dependent and independent variables into account are called Type II regression techniques. In this study, we also compare the performances of Type II and Type I (classical regression techniques that do not take the measurement errors of the independent variable into account) regression techniques for different sample sizes and different shape parameters of the Weibull distribution. The mean square error is used as a performance criterion of each technique. MATLAB 7.02 software is used in the simulation study. As a result, in all conditions, the ordinary least-square (OLS)-bisector regression technique, which bisects the OLS(Y | X) and the OLS(X | Y), shows the best performance.     

High correlation among errors and the efficiency of ordinary least squares in linear models

This note compares ordinary least squares (OLS) and Gauss-Markov (GM) estimates of regression parameters in linear models when errors are homoscedastic but otherwise arbitrary. It is shown that the efficiency of OLS relative to GM depends crucially on the underlying regressor matrix. This extends and qualifies previous results (Krämer 1980), where errors were confined to be first-order autoregressive. In particular, whenever there is a constant in the regression, it is shown that OLS has limiting efficiency of 1 as correlation increases also in the general case.     

A class of generalized ridge estimators

Presence of collinearity among the explanatory variables results in larger standard errors of parameters estimated. When multicollinearity is present among the explanatory variables, the ordinary least-square (OLS) estimators tend to be unstable due to larger variance of the estimators of the regression coefficients. As alternatives to OLS estimators few ridge estimators are available in the literature. This article presents some of the popular ridge estimators and attempts to provide (i) a generalized class of ridge estimators and (ii) a modified ridge estimator. The performance of the proposed estimators is investigated with the help of Monte Carlo simulation technique. Simulation results indicate that the suggested estimators perform better than the ordinary least-square (OLS) estimators and other estimators considered in this article.     

Robust surface estimation in multi-response multistage statistical optimization problems

As the ordinary least squares (OLS) method is very sensitive to outliers as well as to correlated responses, a robust coefficient estimation method is proposed in this paper for multi-response surfaces in multistage processes based on M-estimators. In this approach, experimental designs are used in which the intermediate response variables may act as covariates in the next stages. The performances of both the ordinary multivariate OLS and the proposed robust multi-response surface approach are analyzed and compared through extensive simulation experiments. Sum of the squared errors in estimating the regression coefficients reveals the efficiency of the proposed robust approach.     

A simulation study on SPSS ridge regression and ordinary least squares regression procedures for multicollinearity data

This study compares the SPSS ordinary least squares (OLS) regression and ridge regression procedures in dealing with multicollinearity data. The LS regression method is one of the most frequently applied statistical procedures in application. It is well documented that the LS method is extremely unreliable in parameter estimation while the independent variables are dependent (multicollinearity problem). The Ridge Regression procedure deals with the multicollinearity problem by introducing a small bias in the parameter estimation. The application of Ridge Regression involves the selection of a bias parameter and it is not clear if it works better in applications. This study uses a Monte Carlo method to compare the results of OLS procedure with the Ridge Regression procedure in SPSS.     

Bias From Censored Regressors

We study the bias that arises from using censored regressors in estimation of linear models. We present results on bias in ordinary least aquares (OLS) regression estimators with exogenous censoring and in instrumental variable (IV) estimators when the censored regressor is endogenous. Bound censoring such as top-coding results in expansion bias, or effects that are too large. Independent censoring results in bias that varies with the estimation method—attenuation bias in OLS estimators and expansion bias in IV estimators. Severe biases can result when there are several regressors and when a 0–1 variable is used in place of a continuous regressor.     

Asymptotic properties for LS estimators in EV regression model with dependent errors

In this paper, we establish the strong consistency and asymptotic normality for the least square (LS) estimators in simple linear errors-in-variables (EV) regression models when the errors form a stationary α-mixing sequence of random variables. The quadratic-mean consistency is also considered.     

Estimation of linear models for field experiments

General mixed linear models for experiments conducted over a series of sltes and/or years are described. The ordinary least squares (OLS) estlmator is simple to compute, but is not the best unbiased estimator. Also, the usuaL formula for the varlance of the OLS estimator is not correct and seriously underestimates the true variance. The best linear unbiased estimator is the generalized least squares (GLS) estimator. However, t requires an inversion of the variance-covariance matrix V, whlch is usually of large dimension. Also, in practice, V is unknown.

We presented an estlmator [Vcirc] of the matrix V using the estimators of variance components [for sites, blocks (sites), etc.]. We also presented a simple transformation of the data, such that an ordinary least squares regression of the transformed data gives the estimated generalized least squares (EGLS) estimator. The standard errors obtained from the transformed regression serve as asymptotic standard errors of the EGLS estimators. We also established that the EGLS estlmator is unbiased.

An example of fitting a linear model to data for 18 sites (environments) located in Brazil is given. One of the site variables (soil test phosphorus) was measured by plot rather than by site and this established the need for a covariance model such as the one used rather than the usual analysis of variance model. It is for this variable that the resulting parameter estimates did not correspond well between the OLS and EGLS estimators. Regression statistics and the analysis of variance for the example are presented and summarized.     

Is regression adjustment supported by the Neyman model for causal inference?

This paper examines both theoretically and empirically whether the common practice of using OLS multivariate regression models to estimate average treatment effects (ATEs) under experimental designs is justified by the Neyman model for causal inference. Using data from eight large U.S. social policy experiments, the paper finds that estimated standard errors and significance levels for ATE estimators are similar under the OLS and Neyman models when baseline covariates are included in the models, even though theory suggests that this may not have been the case. This occurs primarily because treatment effects do not appear to vary substantially across study subjects.     

Analysis of cointegrated models with measurement errors

We study the asymptotic properties of the reduced-rank estimator of error correction models of vector processes observed with measurement errors. Although it is well known that there is no asymptotic measurement error bias when predictor variables are integrated processes in regression models [Phillips BCB, Durlauf SN. Multiple time series regression with integrated processes. Rev Econom Stud. 1986;53:473–495], we systematically investigate the effects of the measurement errors (in the dependent variables as well as in the predictor variables) on the estimation of not only cointegrating vectors but also the speed of the adjustment matrix. Furthermore, we present the asymptotic properties of the estimators. We also obtain the asymptotic distribution of the likelihood ratio test for the cointegrating ranks. We investigate the effects of the measurement errors on estimation and test through a Monte Carlo simulation study.     

Note on a linear model with errors in the variables and with trend

The least squares estimate of the slope parameter of a simple linear model with errors in the variables is typically biased. However the bias vanishes asymptotically for increasing sample size if the regressor variable follows a linear trend. For this case asymptotic expansion formulas for bias and variance of the least squares estimator are derived from exact expressions presented by Richardson and Wu (1970) and certain bounds to these expressions given by Friedmann (1990).     

结构突变趋势平稳过程与随机趋势过程的虚假回归研究

 内容提要：在已有研究的基础上,本文更为深入的研究含有结构突变的趋势平稳变量与随机趋势变量间的虚假回归问题。本文推导出OLS估计下DW统计量、F统计量以及R2的极限分布,并且将回归模型扩展到动态情形下,推导出用于Granger因果检验的F统计量的极限分布;采用Monte Carlo模拟方法分析了数据生成过程的各项参数对各统计量有限样本分布的影响;最后,本文分析了在有限样本下,数据生成过程的各项参数对虚假回归及虚假Granger因果关系发生概率的影响。     

Bayesian composite quantile regression

One advantage of quantile regression, relative to the ordinary least-square (OLS) regression, is that the quantile regression estimates are more robust against outliers and non-normal errors in the response measurements. However, the relative efficiency of the quantile regression estimator with respect to the OLS estimator can be arbitrarily small. To overcome this problem, composite quantile regression methods have been proposed in the literature which are resistant to heavy-tailed errors or outliers in the response and at the same time are more efficient than the traditional single quantile-based quantile regression method. This paper studies the composite quantile regression from a Bayesian perspective. The advantage of the Bayesian hierarchical framework is that the weight of each component in the composite model can be treated as open parameter and automatically estimated through Markov chain Monte Carlo sampling procedure. Moreover, the lasso regularization can be naturally incorporated into the model to perform variable selection. The performance of the proposed method over the single quantile-based method was demonstrated via extensive simulations and real data analysis.     

Forecasting with serially correlated regression models

In this article we investigate the asymptotic and finite-sample properties of predictors of regression models with autocorrelated errors. We prove new theorems associated with the predictive efficiency of generalized least squares (GLS) and incorrectly structured GLS predictors. We also establish the form associated with their predictive mean squared errors as well as the magnitude of these errors relative to each other and to those generated from the ordinary least squares (OLS) predictor. A large simulation study is used to evaluate the finite-sample performance of forecasts generated from models using different corrections for the serial correlation.     

Ridge estimation in regression problems with autocorrelated errors: A monte carlo study

This paper presents the results of a Monte Carlo study of OLS and GLS based adaptive ridge estimators for regression problems in which the independent variables are collinear and the errors are autocorrelated. It studies the effects of degree of collinearity, magnitude of error variance, orientation of the parameter vector and serial correlation of the independent variables on the mean squared error performance of these estimators. Results suggest that such estimators produce greatly improved performance in favorable portions of the parameter space. The GLS based methods are best when the independent variables are also serially correlated.     

Modified profile likelihood estimation for the Weibull regression models in survival analysis

In this study, adjustment of profile likelihood function of parameter of interest in presence of many nuisance parameters is investigated for survival regression models. Our objective is to extend the Barndorff–Nielsen’s technique to Weibull regression models for estimation of shape parameter in presence of many nuisance and regression parameters. We conducted Monte-Carlo simulation studies and a real data analysis, all of which demonstrate and suggest that the modified profile likelihood estimators outperform the profile likelihood estimators in terms of three comparison criterion: mean squared errors, bias and standard errors.     

Fatal shortcomings of stationary AR (1) exogeneous variables in econometric monte carlo tudies of estimators

The paper first shows that the stationary normal AR(1) process (SNAR1), the most frequently used process for generating exogenous variables in econometric Monte Carlo studies, cannot generate realistic exogenous variables, which are generally trended and similar to those generated by ARIMA (p,d,q) process withd≧1 and positive drift (trend). Then, it illustrates that in the context of AR(1) disturbances,trends in exogenous variables can frequently alter the very ranking of two competing estimators, the ordinary least squares estimator (OLS) and the Cochrane-Orcutt estimators (CO). For three common econometric models—a standard regression model, a dynamic model (i.e., a model with a lagged dependent variable), and a seemingly unrelated regression model, OLS becomes superior in many cases. This is so in spite of the fact that the CO estimator in the study utilizes the true value of the first-order autocorrelation coefficient of the disturbances. The message to be derived from these findings should be ccear. If one accepts the fact that most if not all economic time series are trended, and endorses a proposition that the fundamental if not sole purpose of Monte Carlo studies in econometrics should be to provide useful guidelines to practicing econometricians, then, he must not employ SNARl (nor anyother artificially created nontrended series) as a generator of exogenous variables in a Monte Carlo study, at least in the econometrics of autocorrelated disturbances. Alternative methods of generating stochastic exogenous variables that are trended are suggested in the paper. For almost four decades, the principle of the autoregressive transformation of a regression model with first-order autocorrelated disturbances (the Coestimation priciple) has been taken for granted as a method of correcting for the autocorrelation in the disturbances—be it in the two-stage Cochrane—Orcutt estimator, the iterative Cochrane-Orcutt estimator, or an estimator utilizing nonlinear techniques or search procedures. (Comitting the first observation due to transformation is not considered very crucial in general.) The results of the pertinent Monte Carlo studies appear to justify such a procedure only because most studies have employed SNARl exogenous variables, not trended ones. Thus, Monte Carlo experimenters must be blamed, at least partially, for this prevailining malpractice. It is hoped that they will not commit additional sins by not using realistic data in their future experiments.