Minimax regression designs for approximately linear models with autocorrelated errors

We study the construction of regression designs, when the random errors are autocorrelated. Our model of dependence assumes that the spectral density g(ω) of the error process is of the form g(ω) = (1 − α)g₀(ω) + αg₁(ω), where g₀(ω) is uniform (corresponding to uncorrelated errors), α ϵ [0, 1) is fixed, and g₁(ω) is arbitrary. We consider regression responses which are exactly, or only approximately, linear in the parameters. Our main results are that a design which is asymptotically (minimax) optimal for uncorrelated errors retains its optimality under autocorrelation if the design points are a random sample, or a random permutation, of points from this distribution. Our results are then a partial extension of those of Wu (Ann. Statist. 9 (1981), 1168–1177), on the robustness of randomized experimental designs, to the field of regression design.     

Additive nonparametric regression with autocorrelated errors

A Bayesian approach is presented for nonparametric estimation of an additive regression model with autocorrelated errors. Each of the potentially non-linear components is modelled as a regression spline using many knots, while the errors are modelled by a high order stationary autoregressive process parameterized in terms of its autocorrelations. The distribution of significant knots and partial autocorrelations is accounted for using subset selection. Our approach also allows the selection of a suitable transformation of the dependent variable. All aspects of the model are estimated simultaneously by using the Markov chain Monte Carlo method. It is shown empirically that the approach proposed works well on several simulated and real examples.     

On the asymptotic bias of OLS in dynamic regression models with autocorrelated errors

It is well-known that Ordinary Least Squares (OLS) yields inconsistent estimates if applied to a regression equation with lagged dependent variables and correlated errors. Bias expressions which appear in the literature usually assume the exogenous variables to be non-stochastic. Due to this assumption the numerical sizes of these expressions cannot be determined. Further, the analysis is mostly restricted to very simple models. In this paper the problem of calculating the asymptotic bias of OLS is generalized to stationary dynamic regression models, where the errors follow a stationary ARMA process. A general bias expression is derived and a method is introduced by which its actual size can be computed numerically.     

Parameter changes in a regression model with autocorrelated errors

This study is a Bayesian analysis of a regression model with autocorrelated errors which exhibits one change in the regression parameters and where the autocorrelation parameter is unknown

Using a normal-gamma prior for all the parameters except the shift point which has a uniform distribution, the marginal posterior distribution of the regression parameters, the shift point and the precision of the errors is found. It is important to know where the shift occurred thus the main emphasis is with the posterior distribution of the shift point

A numerical study assesses the effect of the values of the shift point and the magnitude of the shift on the posterior distribution of the shift point. The posterior distribution of the shift point is more sensitive to change, which occurs in the middle of the observations than to one which occurs at an extreme of the data.     

A Bayesian multiple structural change regression model with autocorrelated errors

This paper develops a new Bayesian approach to change-point modeling that allows the number of change-points in the observed autocorrelated times series to be unknown. The model we develop assumes that the number of change-points have a truncated Poisson distribution. A genetic algorithm is used to estimate a change-point model, which allows for structural changes with autocorrelated errors. We focus considerable attention on the construction of autocorrelated structure for each regime and for the parameters that characterize each regime. Our techniques are found to work well in the simulation with a few change-points. An empirical analysis is provided involving the annual flow of the Nile River and the monthly total energy production in South Korea to lead good estimates for structural change-points.     

Penalized weighted composite quantile regression in the linear regression model with heavy-tailed autocorrelated errors

In this paper, a penalized weighted composite quantile regression estimation procedure is proposed to estimate unknown regression parameters and autoregression coefficients in the linear regression model with heavy-tailed autoregressive errors. Under some conditions, we show that the proposed estimator possesses the oracle properties. In addition, we introduce an iterative algorithm to achieve the proposed optimization problem, and use a data-driven method to choose the tuning parameters. Simulation studies demonstrate that the proposed new estimation method is robust and works much better than the least squares based method when there are outliers in the dataset or the autoregressive error distribution follows heavy-tailed distributions. Moreover, the proposed estimator works comparably to the least squares based estimator when there are no outliers and the error is normal. Finally, we apply the proposed methodology to analyze the electricity demand dataset.     

A method for parameter estimation of non-linear regression with autocorrelated errors

For a class of non-linear models with stationary dependent residuals an estimating procedure is introduced and its statistical properties are derived. This procedure is useful when no basis exists for assuming a specific parametric model for the error process. For application of the procedure a two step iterative method is described and a small simulation study is performed.     

Cross validation of ridge regression estimator in autocorrelated linear regression models

ABSTRACT

In this paper, we investigated the cross validation measures, namely OCV, GCV and Cp under the linear regression models when the error structure is autocorrelated and regressor data are correlated. The best performed ridge regression estimator is obtained by getting the optimal ridge parameter so as to minimize these measures. A Monte Carlo simulation study is given to see how the optimal ridge parameter is affected by autocorrelation and the strength of multicollinearity.     

Ridge estimation in regression problems with autocorrelated errors: A monte carlo study

This paper presents the results of a Monte Carlo study of OLS and GLS based adaptive ridge estimators for regression problems in which the independent variables are collinear and the errors are autocorrelated. It studies the effects of degree of collinearity, magnitude of error variance, orientation of the parameter vector and serial correlation of the independent variables on the mean squared error performance of these estimators. Results suggest that such estimators produce greatly improved performance in favorable portions of the parameter space. The GLS based methods are best when the independent variables are also serially correlated.     

Weighted denoised minimum distance estimation in a regression model with autocorrelated measurement errors

This paper deals with the linear regression model with measurement errors in both response and covariates. The variables are observed with errors together with an auxiliary variable, such as time, and the errors in response are autocorrelated. We propose a weighted denoised minimum distance estimator (WDMDE) for the regression coefficients. The consistency, asymptotic normality, and strong convergence rate of the WDMDE are proved. Compared with the usual denoised least squares estimator (DLSE) in the previous literature, the WDMDE is asymptotically more efficient in the sense of having smaller variances. It also avoids undersmoothing the regressor functions over the auxiliary variable, so that data-driven optimal choice of the bandwidth can be used. Furthermore, we consider the fitting of the error structure, construct the estimators of the autocorrelation coefficients and the error variances, and derive their large-sample properties. Simulations are conducted to examine the finite sample performance of the proposed estimators, and an application of our methodology to analyze a set of real data is illustrated as well.     

Tests of heteroscedasticity and correlation in multivariate t regression models with AR and ARMA errors

Heteroscedasticity checking in regression analysis plays an important role in modelling. It is of great interest when random errors are correlated, including autocorrelated and partial autocorrelated errors. In this paper, we consider multivariate t linear regression models, and construct the score test for the case of AR(1) errors, and ARMA(s,d) errors. The asymptotic properties, including asymptotic chi-square and approximate powers under local alternatives of the score tests, are studied. Based on modified profile likelihood, the adjusted score test is also developed. The finite sample performance of the tests is investigated through Monte Carlo simulations, and also the tests are illustrated with two real data sets.     

Standard errors for the Laspeyres index number with autocorrelated error models

This paper deals with the stochastic approach to Laspeyres price index number with the assumption of serial correlation of orders 1 and 2. The first round of estimation provides the estimates of Laspeyres index numbers in the presence of serial correlation assuming that variance is independent of time. In the second round of estimation, we use the weighted least square approach to derive the standard errors of Laspeyres index number assuming variance is dependent on time. These standard errors are linked to the variability of relative prices and are simple to evaluate. It shows that the larger index numbers are expected to estimate with less degree of precision. The results are illustrated with price data of Pakistan.     

The relative efficiency of OLS in the linear regression model with spatially autocorrelated errors

The relative efficiency of the OLS-estimator in the linear regression model given spatially autocorrelated errors is considered. A theorem of Krämer and Donninger (1987) is shown to be wrong and a corrected proof of this result is given under an additional assumption.     

Heteroscedastic regression models with non-normally distributed errors

Although regression estimates are quite robust to slight departure from normality, symmetric prediction intervals assuming normality can be highly unsatisfactory and problematic if the residuals have a skewed distribution. For data with distributions outside the class covered by the Generalized Linear Model, a common way to handle non-normality is to transform the response variable. Unfortunately, transforming the response variable often destroys the theoretical or empirical functional relationship connecting the mean of the response variable to the explanatory variables established on the original scale. Further complication arises if a single transformation cannot both stabilize variance and attain normality. Furthermore, practitioners also find the interpretation of highly transformed data not obvious and often prefer an analysis on the original scale. The present paper presents an alternative approach for handling simultaneously heteroscedasticity and non-normality without resorting to data transformation. Unlike classical approaches, the proposed modeling allows practitioners to formulate the mean and variance relationships directly on the original scale, making data interpretation considerably easier. The modeled variance relationship and form of non-normality in the proposed approach can be easily examined through a certain function of the standardized residuals. The proposed method is seen to remain consistent for estimating the regression parameters even if the variance function is misspecified. The method along with some model checking techniques is illustrated with a real example.     

Ridge regression estimators in the linear regression models with non-spherical errors

The present paper considers a family of ordinary ridge regression estimators in the linear regression model when the disturbances covariance matrix depends upon a few unknown parameters. An asymptotic expansion for the distribution of the ridge regression estimator is developed and under the quadratic loss function its asymptotic risk is compared with that of the feasible GLS estimator.     

Estimation in regression models with stationary,dependent errors

Often the unknown covariance structure of a stationary, dependent, Gaussian error sequence can be simply parametrised. The error sequence can either be directly observed or observed only through a random sequence containing a deterministic regression model. The method of scoring is used here, in conjunction with recursive estimation techniques, to effect the maximum likelihood estimation of the covariance parameters. Sequences of recursive residuals, useful in model diagnostics and data analysis, are obtained in the estimation procedure.     

Efficiency analysis of ten estimation procedures for quantitative linear models with autocorrelated errors

Many estimation procedures for quantitative linear models with autocorrelated errors have been proposed in the literature. A number of these procedures have been compared in various ways for different sample sizes and autocorrelation parameters values and for structured or random explanatory vaiables. In this paper, we revisit three situations that were considered to some extent in previous studies, by comparing ten estimation procedures: Ordinary Least Squares (OLS), Generalized Least Squares (GLS), estimated Generalized Least Squares (six procedures), Maximum Likelihood (ML), and First Differences (FD). The six estimated GLS procedures and the ML procedure differ in the way the error autocovariance matrix is estimated. The three situations can be defined as follows: Case 1, the explanatory variable x in the simple linear regression is fixed; Case 2,x is purely random; and Case 3x is first-order autoregressive. Following a theoretical presentation, the ten estimation procedures are compared in a Monte Carlo study conducted in the time domain, where the errors are first-order autoregressive in Cases 1-3. The measure of comparison for the estimation procedures is their efficiency relative to OLS. It is evaluated as a function of the time series length and the magnitude and sign of the error autocorrelation parameter. Overall, knowledge of the model of the time series process generating the errors enhances efficiency in estimated GLS. Differences in the efficiency of estimation procedures between Case 1 and Cases 2 and 3 as well as differences in efficiency among procedures in a given situation are observed and discussed.     

The consistency of model selection for dynamic Semi-varying coefficient models with autocorrelated errors

The consistency of model selection criterion BIC has been well and widely studied for many nonlinear regression models. However, few of them had considered models with lag variables as regressors and auto-correlated errors in time series settings, which is common in both linear and nonlinear time series modeling. This paper studies a dynamic semi-varying coefficient model with ARMA errors, using an approach based on spectrum analysis of time series. The consistency property of the proposed model selection criteria is established and an implementation procedure of model selection is proposed for practitioners. Simulation studies have also been conducted to numerically show the consistency property.