The Performance of Robust Two-Stage Estimator in Nonlinear Regression with Autocorrelated Error

Some statistics practitioners often ignore the underlying assumptions when analyzing a real data and employ the Nonlinear Least Squares (NLLS) method to estimate the parameters of a nonlinear model. In order to make reliable inferences about the parameters of a model, require that the underlying assumptions, especially the assumption that the errors are independent, are satisfied. However, in a real situation, we may encounter dependent error terms which prone to produce autocorrelated errors. A two-stage estimator (CTS) has been developed to remedy this problem. Nevertheless, it is now evident that the presence of outliers have an unduly effect on the least squares estimates. We expect that the CTS is also easily affected by outliers since it is based on the least squares estimator, which is not robust. In this article, we propose a Robust Two-Stage (RTS) procedure for the estimation of the nonlinear regression parameters in the situation where autocorrelated errors come together with the existence of outliers. The numerical example and simulation study signify that the RTS is more efficient than the NLLS and the CTS methods.     

A Reinforced Randomized Block Design with Correlated Errors

In the analysis of variance the errors are assumed to be independent which may not always be true. In this article, a restricted randomized block design under compound autocorrelated structure is considered. A method of analysis for such a randomized block design has been developed. It provides confidence ellipsoid and confidence interval of a set of estimable functions and an estimable function of treatment parameters, respectively. Analogous to Scheffe's method, the multiple comparison technique of judging all possible treatment contrasts has been derived. In the process, a method of estimation of unknown parameters has been also developed. An example (simulated data) illustrates this approach.     

Estimation of regression parameters in generalized linear models for cluster correlated data with measurement error

Liang and Zeger (1986) introduced a class of estimating equations that gives consistent estimates of regression parameters and of their asymptotic variances in the class of generalized linear models for cluster correlated data. When the independent variables or covariates in such models are subject to measurement errors, the parameter estimates obtained from these estimating equations are no longer consistent. To correct for the effect of measurement errors, an estimator with smaller asymptotic bias is constructed along the lines of Stefanski (1985), assuming that the measurement error variance is either known or estimable. The asymptotic distribution of the bias-corrected estimator and a consistent estimator of its asymptotic variance are also given. The special case of a binary logistic regression model is studied in detail. For this case, methods based on conditional scores and quasilikelihood are also extended to cluster correlated data. Results of a small simulation study on the performance of the proposed estimators and associated tests of hypotheses are reported.     

Information criteria in identifying regression models

This paper proposes to use information criteria to discriminate the standard regression model from error components models, heteroskedastic models, or models with autocorrelated errors.     

Forecasting with serially correlated regression models

In this article we investigate the asymptotic and finite-sample properties of predictors of regression models with autocorrelated errors. We prove new theorems associated with the predictive efficiency of generalized least squares (GLS) and incorrectly structured GLS predictors. We also establish the form associated with their predictive mean squared errors as well as the magnitude of these errors relative to each other and to those generated from the ordinary least squares (OLS) predictor. A large simulation study is used to evaluate the finite-sample performance of forecasts generated from models using different corrections for the serial correlation.     

Correlated random effects regression analysis for a log-normally distributed variable

In regression analysis, it is assumed that the response (or dependent variable) distribution is Normal, and errors are homoscedastic and uncorrelated. However, in practice, these assumptions are rarely satisfied by a real data set. To stabilize the heteroscedastic response variance, generally, log-transformation is suggested. Consequently, the response variable distribution approaches nearer to the Normal distribution. As a result, the model fit of the data is improved. Practically, a proper (seems to be suitable) transformation may not always stabilize the variance, and the response distribution may not reduce to Normal distribution. The present article assumes that the response distribution is log-normal with compound autocorrelated errors. Under these situations, estimation and testing of hypotheses regarding regression parameters have been derived. From a set of reduced data, we have derived the best linear unbiased estimators of all the regression coefficients, except the intercept which is often unimportant in practice. Unknown correlation parameters have been estimated. In this connection, we have derived a test rule for testing any set of linear hypotheses of the unknown regression coefficients. In addition, we have developed the confidence ellipsoids of a set of estimable functions of regression coefficients. For the fitted regression equation, an index of fit has been proposed. A simulated study illustrates the results derived in this report.     

Weighted denoised minimum distance estimation in a regression model with autocorrelated measurement errors

This paper deals with the linear regression model with measurement errors in both response and covariates. The variables are observed with errors together with an auxiliary variable, such as time, and the errors in response are autocorrelated. We propose a weighted denoised minimum distance estimator (WDMDE) for the regression coefficients. The consistency, asymptotic normality, and strong convergence rate of the WDMDE are proved. Compared with the usual denoised least squares estimator (DLSE) in the previous literature, the WDMDE is asymptotically more efficient in the sense of having smaller variances. It also avoids undersmoothing the regressor functions over the auxiliary variable, so that data-driven optimal choice of the bandwidth can be used. Furthermore, we consider the fitting of the error structure, construct the estimators of the autocorrelation coefficients and the error variances, and derive their large-sample properties. Simulations are conducted to examine the finite sample performance of the proposed estimators, and an application of our methodology to analyze a set of real data is illustrated as well.     

ON SPLINE SMOOTHING WITH AUTOCORRELATED ERRORS

The generalised cross-validation criterion for choosing the degree of smoothing in spline regression is extended to accommodate an autocorrelated error sequence. It is demonstrated via simulation that the minimum generalised cross-validation smoothing spline is an inconsistent estimator in the presence of autocorrelated errors and that ignoring even moderate autocorrelation structure can seriously affect the performance of the cross-validated smoothing spline. The method of penalised maximum likelihood is used to develop an efficient algorithm for the case in which the autocorrelation decays exponentially. An application of the method to a published data-set is described. The method does not require the data to be equally spaced in time.     

A measure of robust slope-rotatability for second-order response surface experimental designs

In response surface methodology, rotatability and slope-rotatability are natural and highly desirable properties for second-order regression models. In this paper a measure of robust slope-rotatability for second-order response surface designs with a general correlated error structure is developed and illustrated with different examples for autocorrelated error structure.     

Tests of heteroscedasticity and correlation in multivariate t regression models with AR and ARMA errors

Heteroscedasticity checking in regression analysis plays an important role in modelling. It is of great interest when random errors are correlated, including autocorrelated and partial autocorrelated errors. In this paper, we consider multivariate t linear regression models, and construct the score test for the case of AR(1) errors, and ARMA(s,d) errors. The asymptotic properties, including asymptotic chi-square and approximate powers under local alternatives of the score tests, are studied. Based on modified profile likelihood, the adjusted score test is also developed. The finite sample performance of the tests is investigated through Monte Carlo simulations, and also the tests are illustrated with two real data sets.     

A class of finite mixture of quantile regressions with its applications

Mixture of linear regression models provide a popular treatment for modeling nonlinear regression relationship. The traditional estimation of mixture of regression models is based on Gaussian error assumption. It is well known that such assumption is sensitive to outliers and extreme values. To overcome this issue, a new class of finite mixture of quantile regressions (FMQR) is proposed in this article. Compared with the existing Gaussian mixture regression models, the proposed FMQR model can provide a complete specification on the conditional distribution of response variable for each component. From the likelihood point of view, the FMQR model is equivalent to the finite mixture of regression models based on errors following asymmetric Laplace distribution (ALD), which can be regarded as an extension to the traditional mixture of regression models with normal error terms. An EM algorithm is proposed to obtain the parameter estimates of the FMQR model by combining a hierarchical representation of the ALD. Finally, the iterated weighted least square estimation for each mixture component of the FMQR model is derived. Simulation studies are conducted to illustrate the finite sample performance of the estimation procedure. Analysis of an aphid data set is used to illustrate our methodologies.     

Cross validation of ridge regression estimator in autocorrelated linear regression models

ABSTRACT

In this paper, we investigated the cross validation measures, namely OCV, GCV and Cp under the linear regression models when the error structure is autocorrelated and regressor data are correlated. The best performed ridge regression estimator is obtained by getting the optimal ridge parameter so as to minimize these measures. A Monte Carlo simulation study is given to see how the optimal ridge parameter is affected by autocorrelation and the strength of multicollinearity.     

On choosing estimators'in a simple linear errors-in-variables model

This paper focuses on studying the accuracy of two well-known estimators in a simple errors-in-variables model, the ordinary least squares and the corrected least squares estimator. As a measure of accuracy of the estimators, the mean squared error is adopted. While Ketellapper (1983) addressed this issue for the case where the error of measurement in the independent variable is known, the present article is concerned with this comparison for the case where the ratio of the error variances is known. Comparison of the mean squared errors of the above estimators leads to a simple rule involving quantities estimable from the data, which can be used for deciding which of the two to be preferred on the basis of higher accuracy.     

Theory & Methods: Krige,Smooth, Both or Neither?

Both kriging and non-parametric regression smoothing can model a non-stationary regression function with spatially correlated errors. However comparisons have mainly been based on ordinary kriging and smoothing with uncorrelated errors. Ordinary kriging attributes smoothness of the response to spatial autocorrelation whereas non-parametric regression attributes trends to a smooth regression function. For spatial processes it is reasonable to suppose that the response is due to both trend and autocorrelation. This paper reviews methodology for non-parametric regression with autocorrelated errors which is a natural compromise between the two methods. Re-analysis of the one-dimensional stationary spatial data of Laslett (1994) and a clearly non-stationary time series demonstrates the rather surprising result that for these data, ordinary kriging outperforms more computationally intensive models including both universal kriging and correlated splines for spatial prediction. For estimating the regression function, non-parametric regression provides adaptive estimation, but the autocorrelation must be accounted for in selecting the smoothing parameter.     

Nonparametric Inference for Time-Varying Coefficient Quantile Regression

The article considers nonparametric inference for quantile regression models with time-varying coefficients. The errors and covariates of the regression are assumed to belong to a general class of locally stationary processes and are allowed to be cross-dependent. Simultaneous confidence tubes (SCTs) and integrated squared difference tests (ISDTs) are proposed for simultaneous nonparametric inference of the latter models with asymptotically correct coverage probabilities and Type I error rates. Our methodologies are shown to possess certain asymptotically optimal properties. Furthermore, we propose an information criterion that performs consistent model selection for nonparametric quantile regression models of nonstationary time series. For implementation, a wild bootstrap procedure is proposed, which is shown to be robust to the dependent and nonstationary data structure. Our method is applied to studying the asymmetric and time-varying dynamic structures of the U.S. unemployment rate since the 1940s. Supplementary materials for this article are available online.     

Empirical likelihood estimation for linear regression models with AR(p) error terms with numerical examples

Linear regression models are useful statistical tools to analyze data sets in different fields. There are several methods to estimate the parameters of a linear regression model. These methods usually perform under normally distributed and uncorrelated errors. If error terms are correlated the Conditional Maximum Likelihood (CML) estimation method under normality assumption is often used to estimate the parameters of interest. The CML estimation method is required a distributional assumption on error terms. However, in practice, such distributional assumptions on error terms may not be plausible. In this paper, we propose to estimate the parameters of a linear regression model with autoregressive error term using Empirical Likelihood (EL) method, which is a distribution free estimation method. A small simulation study is provided to evaluate the performance of the proposed estimation method over the CML method. The results of the simulation study show that the proposed estimators based on EL method are remarkably better than the estimators obtained from CML method in terms of mean squared errors (MSE) and bias in almost all the simulation configurations. These findings are also confirmed by the results of the numerical and real data examples.     

Discrete M-robust designs for regression models

M-robust designs are defined and constructed for misspecified linear regression models with possibly autocorrelated errors on a discrete design space. These designs minimize the mean-squared errors if linear regression models are correct with uncorrelated errors, subject to two robust constraints which control the change of the bias and the change of variance under model departures. Simulated annealing algorithm is applied to construct M-robust designs. Examples are given to show M-robust designs and compare them with minimax robust designs.     

Seasonal patterns of fertility measures: A Bayesian approach

Becker (1981) presents some theory about related measures of fertility. He SUMMARY compares his theoretical predictions with observed relationships found in a set of data collected in Bangladesh. In general, he finds good agreement. In this paper, we reanalyse the data using Bayesian methods. In particular, we use Gibbs sampling to fit trigonometric regression models with autocorrelated errors. The results are generally in agreement with Becker's. However, evidence from one of the autocorrelation parameters and a residual analysis casts some doubt on whether the basic cosine model which is assumed fits the data well.     

A robust estimation method for the linear regression model parameters with correlated error terms and outliers

Independence of error terms in a linear regression model, often not established. So a linear regression model with correlated error terms appears in many applications. According to the earlier studies, this kind of error terms, basically can affect the robustness of the linear regression model analysis. It is also shown that the robustness of the parameters estimators of a linear regression model can stay using the M-estimator. But considering that, it acquires this feature as the result of establishment of its efficiency. Whereas, it has been shown that the minimum Matusita distance estimators, has both features robustness and efficiency at the same time. On the other hand, because the Cochrane and Orcutt adjusted least squares estimators are not affected by the dependence of the error terms, so they are efficient estimators. Here we are using of a non-parametric kernel density estimation method, to give a new method of obtaining the minimum Matusita distance estimators for the linear regression model with correlated error terms in the presence of outliers. Also, simulation and real data study both are done for the introduced estimation method. In each case, the proposed method represents lower biases and mean squared errors than the other two methods.KEYWORDS: Robust estimation method, minimum Matusita distance estimation method, non-parametric kernel density estimation method, correlated error terms, outliers     

AN EXAMINATION OF ESTIMATED RESIDUALS IN A REGRESSION WITH AN INFINITE ORDER PARAMETRIC MODEL

We consider a linear regression with the error term that obeys an autoregressive model of infinite order and estimate parameters of the models. The parameters of the autoregressive model should be estimated based on estimated residuals obtained by means of the method of ordinary least squares, because the errors are unobservable. The consistency of the coefficients, variance and spectral density of the model obeyed by the error term is shown. Further, we estimate the coefficients of the linear regression by means of the method of estimated generalized least squares. We also show the consistency of the estimator.