Theory & Methods: Krige,Smooth, Both or Neither?

Both kriging and non-parametric regression smoothing can model a non-stationary regression function with spatially correlated errors. However comparisons have mainly been based on ordinary kriging and smoothing with uncorrelated errors. Ordinary kriging attributes smoothness of the response to spatial autocorrelation whereas non-parametric regression attributes trends to a smooth regression function. For spatial processes it is reasonable to suppose that the response is due to both trend and autocorrelation. This paper reviews methodology for non-parametric regression with autocorrelated errors which is a natural compromise between the two methods. Re-analysis of the one-dimensional stationary spatial data of Laslett (1994) and a clearly non-stationary time series demonstrates the rather surprising result that for these data, ordinary kriging outperforms more computationally intensive models including both universal kriging and correlated splines for spatial prediction. For estimating the regression function, non-parametric regression provides adaptive estimation, but the autocorrelation must be accounted for in selecting the smoothing parameter.     

Regression analysis of autocorrelated poisson-distribution) data

A regression model assuming Poisson-dia distributed data. with autocorrelated errors falls into the class of regression models that; have the error structure which is both heteroscedastic and autocorrelated. In general, this class of regression models are not estimable. However, due to the properties of the Poisson distribution that the variance is equal to the mean, this regression model on Poisson-distributed data with autocorrelated. errors is estimable. In this note the special structure of the covarlance matrix of the model with the first order auto-correlated error Is derived utilizing this property, A method based on the least squares method of Frome, Kutner, and Beauchamp (1973), supplemented by steps for handling autocorrelation in studies of time series analysis, nonlinear regression, and econometrics is presented for obtaining generalized least squares estimates for the parameters of the model.     

The effect of Phase I sample size on the run length performance of control charts for autocorrelated data

Traditional control charts assume independence of observations obtained from the monitored process. However, if the observations are autocorrelated, these charts often do not perform as intended by the design requirements. Recently, several control charts have been proposed to deal with autocorrelated observations. The residual chart, modified Shewhart chart, EWMAST chart, and ARMA chart are such charts widely used for monitoring the occurrence of assignable causes in a process when the process exhibits inherent autocorrelation. Besides autocorrelation, one other issue is the unknown values of true process parameters to be used in the control chart design, which are often estimated from a reference sample of in-control observations. Performances of the above-mentioned control charts for autocorrelated processes are significantly affected by the sample size used in a Phase I study to estimate the control chart parameters. In this study, we investigate the effect of Phase I sample size on the run length performance of these four charts for monitoring the changes in the mean of an autocorrelated process, namely an AR(1) process. A discussion of the practical implications of the results and suggestions on the sample size requirements for effective process monitoring are provided.     

Fatal shortcomings of stationary AR (1) exogeneous variables in econometric monte carlo tudies of estimators

The paper first shows that the stationary normal AR(1) process (SNAR1), the most frequently used process for generating exogenous variables in econometric Monte Carlo studies, cannot generate realistic exogenous variables, which are generally trended and similar to those generated by ARIMA (p,d,q) process withd≧1 and positive drift (trend). Then, it illustrates that in the context of AR(1) disturbances,trends in exogenous variables can frequently alter the very ranking of two competing estimators, the ordinary least squares estimator (OLS) and the Cochrane-Orcutt estimators (CO). For three common econometric models—a standard regression model, a dynamic model (i.e., a model with a lagged dependent variable), and a seemingly unrelated regression model, OLS becomes superior in many cases. This is so in spite of the fact that the CO estimator in the study utilizes the true value of the first-order autocorrelation coefficient of the disturbances. The message to be derived from these findings should be ccear. If one accepts the fact that most if not all economic time series are trended, and endorses a proposition that the fundamental if not sole purpose of Monte Carlo studies in econometrics should be to provide useful guidelines to practicing econometricians, then, he must not employ SNARl (nor anyother artificially created nontrended series) as a generator of exogenous variables in a Monte Carlo study, at least in the econometrics of autocorrelated disturbances. Alternative methods of generating stochastic exogenous variables that are trended are suggested in the paper. For almost four decades, the principle of the autoregressive transformation of a regression model with first-order autocorrelated disturbances (the Coestimation priciple) has been taken for granted as a method of correcting for the autocorrelation in the disturbances—be it in the two-stage Cochrane—Orcutt estimator, the iterative Cochrane-Orcutt estimator, or an estimator utilizing nonlinear techniques or search procedures. (Comitting the first observation due to transformation is not considered very crucial in general.) The results of the pertinent Monte Carlo studies appear to justify such a procedure only because most studies have employed SNARl exogenous variables, not trended ones. Thus, Monte Carlo experimenters must be blamed, at least partially, for this prevailining malpractice. It is hoped that they will not commit additional sins by not using realistic data in their future experiments.     

Analyzing the effects of level shifts and temporary changes on the identification of ARIMA models

The presence of outliers in time series gives rise to important effects on the sample autocorrelation coefficients. In the case where these outliers are not adequately treated, their presence causes errors in the identification of the stochastic process generator of the time series under study. In this respect, Chan has demonstrated that, independent of the underlying process of the outlier-free series, a level shift (LS) at the limit (i.e. asymptotically and considering an LS of a sufficiently large size) will lead to the identification of non-stationary processes; with respect to a temporary change (TC), this will lead, again at the limit, to the identification of an AR(1) autoregressive process with a coefficient equal to the dampening factor that defines this TC. The objective of this paper is to analyze, by way of a simulation exercise, how large the LS and TC present in the time series must be for the limiting result to be relevant, in the sense of seriously affecting the instruments used at the identification stage of the ARIMA models, i.e. the sample autocorrelation function and the sample partial autocorrelation function.     

Monitoring the mean of autocorrelated observations with one generally weighted moving average control chart

Traditionally, using a control chart to monitor a process assumes that process observations are normally and independently distributed. In fact, for many processes, products are either connected or autocorrelated and, consequently, obtained observations are autocorrelative rather than independent. In this scenario, applying an independence assumption instead of autocorrelation for process monitoring is unsuitable. This study examines a generally weighted moving average (GWMA) with a time-varying control chart for monitoring the mean of a process based on autocorrelated observations from a first-order autoregressive process (AR(1)) with random error. Simulation is utilized to evaluate the average run length (ARL) of exponentially weighted moving average (EWMA) and GWMA control charts. Numerous comparisons of ARLs indicate that the GWMA control chart requires less time to detect various shifts at low levels of autocorrelation than those at high levels of autocorrelation. The GWMA control chart is more sensitive than the EWMA control chart for detecting small shifts in a process mean.     

On a mixture autoregressive model

We generalize the Gaussian mixture transition distribution (GMTD) model introduced by Le and co-workers to the mixture autoregressive (MAR) model for the modelling of non-linear time series. The models consist of a mixture of K stationary or non-stationary AR components. The advantages of the MAR model over the GMTD model include a more full range of shape changing predictive distributions and the ability to handle cycles and conditional heteroscedasticity in the time series. The stationarity conditions and autocorrelation function are derived. The estimation is easily done via a simple EM algorithm and the model selection problem is addressed. The shape changing feature of the conditional distributions makes these models capable of modelling time series with multimodal conditional distributions and with heteroscedasticity. The models are applied to two real data sets and compared with other competing models. The MAR models appear to capture features of the data better than other competing models do.     

Autocorrelation in real-time continuous monitoring of microenvironments

Interpretation of continuous measurements in microenvironmental studies and exposure assessments can be complicated by autocorrelation, the implications of which are often not fully addressed. We discuss some statistical issues that arose in the analysis of microenvironmental particulate matter concentration data collected in 1998 by the Harvard School of Public Health. We present a simulation study that suggests that Generalized Estimating Equations, a technique often used to adjust for autocorrelation, may produce inflated Type I errors when applied to microenvironmental studies of small or moderate sample size, and that Linear Mixed Effects models may be more appropriate in small-sample settings. Environmental scientists often appeal to longer averaging times to reduce autocorrelation. We explore the functional relationship between averaging time, autocorrelation, and standard errors of both mean and variance, showing that longer averaging times impair statistical inferences about main effects. We conclude that, given widely available techniques that adjust for autocorrelation, longer averaging times may be inappropriate in microenvironmental studies.     

Goodness-of-fit tests for binomial AR(1) processes

The binomial AR(1) model describes a nonlinear process with a first-order autoregressive (AR(1)) structure and a binomial marginal distribution. To develop goodness-of-fit tests for the binomial AR(1) model, we investigate the observed marginal distribution of the binomial AR(1) process, and we tackle its autocorrelation structure. Motivated by the family of power-divergence statistics for handling discrete multivariate data, we derive the asymptotic distribution of certain categorized power-divergence statistics for the case of a binomial AR(1) process. Then we consider Bartlett's formula, which is widely used in time series analysis to provide estimates of the asymptotic covariance between sample autocorrelations, but which is not applicable when the underlying process is nonlinear. Hence, we derive a novel Bartlett-type formula for the asymptotic distribution of the sample autocorrelations of a binomial AR(1) process, which is then applied to develop tests concerning the autocorrelation structure. Simulation studies are carried out to evaluate the size and power of the proposed tests under diverse alternative process models. Several real examples are used to illustrate our methods and findings.     

Influence diagnostics for censored regression models with autoregressive errors

Observations collected over time are often autocorrelated rather than independent, and sometimes include observations below or above detection limits (i.e. censored values reported as less or more than a level of detection) and/or missing data. Practitioners commonly disregard censored data cases or replace these observations with some function of the limit of detection, which often results in biased estimates. Moreover, parameter estimation can be greatly affected by the presence of influential observations in the data. In this paper we derive local influence diagnostic measures for censored regression models with autoregressive errors of order p (hereafter, AR(p)‐CR models) on the basis of the Q‐function under three useful perturbation schemes. In order to account for censoring in a likelihood‐based estimation procedure for AR(p)‐CR models, we used a stochastic approximation version of the expectation‐maximisation algorithm. The accuracy of the local influence diagnostic measure in detecting influential observations is explored through the analysis of empirical studies. The proposed methods are illustrated using data, from a study of total phosphorus concentration, that contain left‐censored observations. These methods are implemented in the R package ARCensReg.     

Seasonal patterns of fertility measures: A Bayesian approach

Becker (1981) presents some theory about related measures of fertility. He SUMMARY compares his theoretical predictions with observed relationships found in a set of data collected in Bangladesh. In general, he finds good agreement. In this paper, we reanalyse the data using Bayesian methods. In particular, we use Gibbs sampling to fit trigonometric regression models with autocorrelated errors. The results are generally in agreement with Becker's. However, evidence from one of the autocorrelation parameters and a residual analysis casts some doubt on whether the basic cosine model which is assumed fits the data well.     

Robustness of measures of common cause sigma in presence of data correlation

Process monitoring in the presence of data correlation is one of the most discussed issues in statistical process control literature over the past decade. However, the attention to retrospective analysis in the presence of data correlation with various common cause sigma estimators is lacking in the literature. Maragah et al. (1992), in an early paper on the retrospective analysis in presence of data correlation, addresses only a single common cause sigma estimator. This paper studies the effect of data correlation on retrospective X-chart with various common cause sigma estimates in stable period of AR(1) Process. This study is carried out with the aim of identifying suitable standard deviation statistic/statistics which is/are robust to the data correlation. This paper also discusses the robustness of common cause sigma estimates for monitoring the data following other time series models, namely ARMA(1,1) and AR(p). Further, the bias characteristics of robust standard deviation estimates have been discussed for the above time-series models. This paper further studies the performance of retrospective X-chart on forecast residuals from various forecasting methods of AR(1) process. The above studies were carried out through simulating the stable period of AR(1), AR(2), stable and invertible period of ARMA(1,1) processes. The average number of false alarms have been considered as a measure of performance. The results of simulation studies have been discussed.     

Tests of heteroscedasticity and correlation in multivariate t regression models with AR and ARMA errors

Heteroscedasticity checking in regression analysis plays an important role in modelling. It is of great interest when random errors are correlated, including autocorrelated and partial autocorrelated errors. In this paper, we consider multivariate t linear regression models, and construct the score test for the case of AR(1) errors, and ARMA(s,d) errors. The asymptotic properties, including asymptotic chi-square and approximate powers under local alternatives of the score tests, are studied. Based on modified profile likelihood, the adjusted score test is also developed. The finite sample performance of the tests is investigated through Monte Carlo simulations, and also the tests are illustrated with two real data sets.     

ON SPLINE SMOOTHING WITH AUTOCORRELATED ERRORS

The generalised cross-validation criterion for choosing the degree of smoothing in spline regression is extended to accommodate an autocorrelated error sequence. It is demonstrated via simulation that the minimum generalised cross-validation smoothing spline is an inconsistent estimator in the presence of autocorrelated errors and that ignoring even moderate autocorrelation structure can seriously affect the performance of the cross-validated smoothing spline. The method of penalised maximum likelihood is used to develop an efficient algorithm for the case in which the autocorrelation decays exponentially. An application of the method to a published data-set is described. The method does not require the data to be equally spaced in time.     

A Robust Control Chart for Monitoring the Mean of an Autocorrelated Process

Residual control charts are frequently used for monitoring autocorrelated processes. In the design of a residual control chart, values of the true process parameters are often estimated from a reference sample of in-control observations by using least squares (LS) estimators. We propose a robust control chart for autocorrelated data by using Modified Maximum Likelihood (MML) estimators in constructing a residual control chart. Average run length (ARL) is simulated for the proposed chart when the underlying process is AR(1). The results show the superiority of the new chart under several situations. Moreover, the chart is robust to plausible deviations from assumed distribution of errors.     

Monitoring correlated processes with binomial marginals

Few approaches for monitoring autocorrelated attribute data have been proposed in the literature. If the marginal process distribution is binomial, then the binomial AR(1) model as a realistic and well-interpretable process model may be adequate. Based on known and newly derived statistical properties of this model, we shall develop approaches to monitor a binomial AR(1) process, and investigate their performance in a simulation study. A case study demonstrates the applicability of the binomial AR(1) model and of the proposed control charts to problems from statistical process control.     

Cross validation of ridge regression estimator in autocorrelated linear regression models

ABSTRACT

In this paper, we investigated the cross validation measures, namely OCV, GCV and Cp under the linear regression models when the error structure is autocorrelated and regressor data are correlated. The best performed ridge regression estimator is obtained by getting the optimal ridge parameter so as to minimize these measures. A Monte Carlo simulation study is given to see how the optimal ridge parameter is affected by autocorrelation and the strength of multicollinearity.     

Bayesian model averaging in longitudinal regression models with AR(1) errors with application to a myopia data set

We propose a new iterative algorithm, called model walking algorithm, to the Bayesian model averaging method on the longitudinal regression models with AR(1) random errors within subjects. The Markov chain Monte Carlo method together with the model walking algorithm are employed. The proposed method is successfully applied to predict the progression rates on a myopia intervention trial in children.     

Statistical Monitoring of Autocorrelated Simple Linear Profiles Based on Principal Components Analysis

In this article, a transformation method using the principal component analysis approach is first applied to remove the existing autocorrelation within each profile in Phase I monitoring of autocorrelated simple linear profiles. This easy-to-use approach is independent of the autocorrelation coefficient. Moreover, since it is a model-free method, it can be used for Phase I monitoring procedures. Then, five control schemes are proposed to monitor the parameters of the profile with uncorrelated error terms. The performances of the proposed control charts are evaluated and are compared through simulation experiments based on different values of autocorrelation coefficient as well as different shift scenarios in the parameters of the profile in terms of probability of receiving an out-of-control signal.     

To be or not to be valid in testing the significance of the slope in simple quantitative linear models with autocorrelated errors

In this article, the validity of procedures for testing the significance of the slope in quantitative linear models with one explanatory variable and first-order autoregressive [AR(1)] errors is analyzed in a Monte Carlo study conducted in the time domain. Two cases are considered for the regressor: fixed and trended versus random and AR(1). In addition to the classical t -test using the Ordinary Least Squares (OLS) estimator of the slope and its standard error, we consider seven t -tests with n-2\,\hbox{df} built on the Generalized Least Squares (GLS) estimator or an estimated GLS estimator, three variants of the classical t -test with different variances of the OLS estimator, two asymptotic tests built on the Maximum Likelihood (ML) estimator, the F -test for fixed effects based on the Restricted Maximum Likelihood (REML) estimator in the mixed-model approach, two t -tests with n - 2 df based on first differences (FD) and first-difference ratios (FDR), and four modified t -tests using various corrections of the number of degrees of freedom. The FDR t -test, the REML F -test and the modified t -test using Dutilleul's effective sample size are the most valid among the testing procedures that do not assume the complete knowledge of the covariance matrix of the errors. However, modified t -tests are not applicable and the FDR t -test suffers from a lack of power when the regressor is fixed and trended ( i.e. , FDR is the same as FD in this case when observations are equally spaced), whereas the REML algorithm fails to converge at small sample sizes. The classical t -test is valid when the regressor is fixed and trended and autocorrelation among errors is predominantly negative, and when the regressor is random and AR(1), like the errors, and autocorrelation is moderately negative or positive. We discuss the results graphically, in terms of the circularity condition defined in repeated measures ANOVA and of the effective sample size used in correlation analysis with autocorrelated sample data. An example with environmental data is presented.