Non-Gaussian Conditional Linear AR(1) Models

This paper gives a general formulation of a non-Gaussian conditional linear AR(1) model subsuming most of the non-Gaussian AR(1) models that have appeared in the literature. It derives some general results giving properties for the stationary process mean, variance and correlation structure, and conditions for stationarity. These results highlight similarities with and differences from the Gaussian AR(1) model, and unify many separate results appearing in the literature. Examples illustrate the wide range of properties that can appear under the conditional linear autoregressive assumption. These results are used in analysing three real datasets, illustrating general methods of estimation, model diagnostics and model selection. In particular, the theoretical results can be used to develop diagnostics for deciding if a time series can be modelled by some linear autoregressive model, and for selecting among several candidate models.     

Nonlinear semiparametric AR(1) model with skew-symmetric innovations

In this paper, we expand a first-order nonlinear autoregressive (AR) model with skew normal innovations. A semiparametric method is proposed to estimate a nonlinear part of model by using the conditional least squares method for parametric estimation and the nonparametric kernel approach for the AR adjustment estimation. Then computational techniques for parameter estimation are carried out by the maximum likelihood (ML) approach using Expectation-Maximization (EM) type optimization and the explicit iterative form for the ML estimators are obtained. Furthermore, in a simulation study and a real application, the accuracy of the proposed methods is verified.     

Covariance analysis and associated spectra for classes of nonstationary processes

In 1951, Cramér introduced a class of nonstationary processes. This broad class of processes contains the important harmonizable and stationary classes of processes. The Cramér class can have additional structure imposed upon it through Cesàro summability considerations. These refined Cramér classes, termed (c,p)-summable Cramér, have recently been considered by Swift (in: M.M. Rao (Ed.), Real and Stochastic Analysis: Recent Advances, CRC Press, Boca Raton, FL, 1997, p. 303). In this paper, the relationship between the (c,p)-summable Cramér classes and the (KF,p) classes of processes introduced by Rao in 1985 is considered. The (KF,p) classes of processes are a generalization of the class of processes considered by Kampé de Feriet and Frenkiel. A continuity theorem for the (KF,p) classes is obtained. This result yields a spectral representation for the (KF,p) classes. Some (KF,p) class processes are shown to arise as the solution to a difference equation obtained from a linear model of a noisy communication channel.     

Inference for linear and nonlinear stable error processes via estimating functions

This paper describes an estimating function approach for parameter estimation in linear and nonlinear times series models with infinite variance stable errors. Joint estimates of location and scale parameters are derived for classes of autoregressive (AR) models and random coefficient autoregressive (RCA) models with stable errors, as well as for AR models with stable autoregressive conditionally heteroscedastic (ARCH) errors. Fast, on-line, recursive parametric estimation for the location parameter based on estimating functions is discussed using simulation studies. A real financial time series is also discussed in some detail.     

Optimal tests in AR (m) time series model

In the univariate framework, two problems of testing the nonlinearity are investigated in Hwang and Basawa. The first one is concerned with the testing problem for a nonlinear class contiguous to the AR(1) process. The second one is focused on the testing problem of the ARCH model contiguous to the AR(1) models. In each case, an efficient test of linearity was obtained, the local asymptotic normality (LAN) was proved, an efficient test of linearity was constructed, and the asymptotic power function was derived. All these results were obtained under the assumption where the parameter of the time series model is assumed to be known. In practical situation, this parameter is unspecified and its estimation induces an error that has an impact on the asymptotic limit distribution. A new method for the good evaluation of this error is introduced and investigated in the present article. Consequently, its application allows us to preserve the local asymptotic optimality with the estimated parameter. An extension to testing in class of ARCH models contiguous to p-order autoregressive processes is obtained. The LAN property plays a fundamental role in the present study.     

Optimal partially balanced nested row-column designs

A partially balanced nested row-column design, referred to as PBNRC, is defined as an arrangement of v treatments in b p × q blocks for which, with the convention that p q, the information matrix for the estimation of treatment parameters is equal to that of the column component design which is itself a partially balanced incomplete block design. In this paper, previously known optimal incomplete block designs, and row-column and nested row-column designs are utilized to develop some methods of constructing optimal PBNRC designs. In particular, it is shown that an optimal group divisible PBNRC design for v = mn kn treatments in p × q blocks can be constructed whenever a balanced incomplete block design for m treatments in blocks of size k each and a group divisible PBNRC design for kn treatments in p × q blocks exist. A simple sufficient condition is given under which a group divisible PBNRC is Ψ_f-better for all f> 0 than the corresponding balanced nested row-column designs having binary blocks. It is also shown that the construction techniques developed particularly for group divisible designs can be generalized to obtain PBNRC designs based on rectangular association schemes.     

Lp-norm spherical distribution

In this paper, the spherical distribution is generalized to the L_p-norm spherical distribution, S(n, p), through the stochastic representation. Then the form of its p.d.f. characterizing the S(n, p) distribution is derived. Marginal, conditional distributions and the moments of S(n, p) distribution are obtained. The order statistics and the invariance property of S(n,p) distribution are also investigated.     

A comparison of LS/ML and GMM estimation in a simple AR(1) model

This paper compares least squares (LS)/maximum likelihood (ML) and generalised method of moments (GMM) estimation in a simple. Gaussian autoregressive of order one (AR(1)) model. First, we show that the usual LS/ML estimator is a corner solution to a general minimisation problem that involves two moment conditions, while the new GMM we devise is not. Secondly, we examine asymptotic and finite sample properties of the new GMM estimator in comparison to the usual LS/ML estimator in a simple AR(1) model. For both stable and unstable (unit root) specifications, we show asymptotic equivalence of the distributions of the two estimators. However, in finite samples, the new GMM estimator performs better.     

Efficiency and Validity Analyses of Two-Stage Estimation Procedures and Derived Testing Procedures in Quantitative Linear Models with AR(1) Errors

Abstract

In a quantitative linear model with errors following a stationary Gaussian, first-order autoregressive or AR(1) process, Generalized Least Squares (GLS) on raw data and Ordinary Least Squares (OLS) on prewhitened data are efficient methods of estimation of the slope parameters when the autocorrelation parameter of the error AR(1) process, ρ, is known. In practice, ρ is generally unknown. In the so-called two-stage estimation procedures, ρ is then estimated first before using the estimate of ρ to transform the data and estimate the slope parameters by OLS on the transformed data. Different estimators of ρ have been considered in previous studies. In this article, we study nine two-stage estimation procedures for their efficiency in estimating the slope parameters. Six of them (i.e., three noniterative, three iterative) are based on three estimators of ρ that have been considered previously. Two more (i.e., one noniterative, one iterative) are based on a new estimator of ρ that we propose: it is provided by the sample autocorrelation coefficient of the OLS residuals at lag 1, denoted r(1). Lastly, REstricted Maximum Likelihood (REML) represents a different type of two-stage estimation procedure whose efficiency has not been compared to the others yet. We also study the validity of the testing procedures derived from GLS and the nine two-stage estimation procedures. Efficiency and validity are analyzed in a Monte Carlo study. Three types of explanatory variable x in a simple quantitative linear model with AR(1) errors are considered in the time domain: Case 1, x is fixed; Case 2, x is purely random; and Case 3, x follows an AR(1) process with the same autocorrelation parameter value as the error AR(1) process. In a preliminary step, the number of inadmissible estimates and the efficiency of the different estimators of ρ are compared empirically, whereas their approximate expected value in finite samples and their asymptotic variance are derived theoretically. Thereafter, the efficiency of the estimation procedures and the validity of the derived testing procedures are discussed in terms of the sample size and the magnitude and sign of ρ. The noniterative two-stage estimation procedure based on the new estimator of ρ is shown to be more efficient for moderate values of ρ at small sample sizes. With the exception of small sample sizes, REML and its derived F-test perform the best overall. The asymptotic equivalence of two-stage estimation procedures, besides REML, is observed empirically. Differences related to the nature, fixed or random (uncorrelated or autocorrelated), of the explanatory variable are also discussed.     

On Prediction for Correlated Domains in Longitudinal Surveys

In the survey sampling estimation or prediction of both population’s and subopulation’s (domain’s) characteristics is one of the key issues. In the case of the estimation or prediction of domain’s characteristics one of the problems is looking for additional sources of information that can be used to increase the accuracy of estimators or predictors. One of these sources may be spatial and temporal autocorrelation. Due to the mean squared error (MSE) estimation, the standard assumption is that random variables are independent for population elements from different domains. If the assumption is taken into account, spatial correlation may be assumed only inside domains. In the paper, we assume some special case of the linear mixed model with two random components that obey assumptions of the first-order spatial autoregressive model SAR(1) (but inside groups of domains instead of domains) and first-order temporal autoregressive model AR(1). Based on the model, the empirical best linear unbiased predictor will be proposed together with an estimator of its MSE taking the spatial correlation between domains into account.     

Some simulations and applications of forecasting long-memory time-series models

In this paper, we show some results of forecasting based on the ARFIMA(p,d,q) and ARIMA(p,d,q) models. We show, by simulation, that the technique of forecasting of the ARIMA(p,d,q) model can also be used when d is fractional, i.e., for the ARFIMA(p,d,q) model. We also conduct a simulation study to compare the two estimators of d obtained through regression methods. They are used in the hypothesis test to decide whether or not the series has long memory property and are compared on the basis of their k-step ahead forecast errors. The properties of long-memory models are also investigated using an actual set of data.     

Pitfalls in the Estimation of a Differenced Model

This article assesses the potential magnitude of the loss of estimation efficiency caused by the adoption of a differenced model when the disturbances of the original (levels) linear regression model follow either a stable (autoregressive) AR(1) process or a fixed start-up random-walk process (hence no filtering is necessary from the standpoint of estimation). The magnitude of the loss, which can be quite large, is found to be affected by both the form of the original model (homogeneous or nonhomogeneous) and the sign and magnitude of the autocorrelation coefficient of the AR(1) disturbance, as well as by the nature of the exogenous variable (smoothly trended or not).     

Bias Reduction through First-order Mean Correction, Bootstrapping and Recursive Mean Adjustment

Standard methods of estimation for autoregressive models are known to be biased in finite samples, which has implications for estimation, hypothesis testing, confidence interval construction and forecasting. Three methods of bias reduction are considered here: first-order bias correction, FOBC, where the total bias is approximated by the O(T^-1) bias; bootstrapping; and recursive mean adjustment, RMA. In addition, we show how first-order bias correction is related to linear bias correction. The practically important case where the AR model includes an unknown linear trend is considered in detail. The fidelity of nominal to actual coverage of confidence intervals is also assessed. A simulation study covers the AR(1) model and a number of extensions based on the empirical AR(p) models fitted by Nelson & Plosser (1982). Overall, which method dominates depends on the criterion adopted: bootstrapping tends to be the best at reducing bias, recursive mean adjustment is best at reducing mean squared error, whilst FOBC does particularly well in maintaining the fidelity of confidence intervals.     

Bayesian analysis of a general growth curve model with predictions using power transformations and AR(1) autoregressive dependence

In this paper, we consider a Bayesian analysis of the unbalanced (general) growth curve model with AR(1) autoregressive dependence, while applying the Box-Cox power transformations. We propose exact, simple and Markov chain Monte Carlo approximate parameter estimation and prediction of future values. Numerical results are illustrated with real and simulated data.     

Expected estimating equations to accommodate covariate measurement error

Estimating equations which are not necessarily likelihood-based score equations are becoming increasingly popular for estimating regression model parameters. This paper is concerned with estimation based on general estimating equations when true covariate data are missing for all the study subjects, but surrogate or mismeasured covariates are available instead. The method is motivated by the covariate measurement error problem in marginal or partly conditional regression of longitudinal data. We propose to base estimation on the expectation of the complete data estimating equation conditioned on available data. The regression parameters and other nuisance parameters are estimated simultaneously by solving the resulting estimating equations. The expected estimating equation (EEE) estimator is equal to the maximum likelihood estimator if the complete data scores are likelihood scores and conditioning is with respect to all the available data. A pseudo-EEE estimator, which requires less computation, is also investigated. Asymptotic distribution theory is derived. Small sample simulations are conducted when the error process is an order 1 autoregressive model. Regression calibration is extended to this setting and compared with the EEE approach. We demonstrate the methods on data from a longitudinal study of the relationship between childhood growth and adult obesity.     

COMPARING TESTS OF AUTOREGRESSIVE VERSUS MOVING AVERAGE ERRORS IN REGRESSION MODELS USING BAHADUR’S ASYMPTOTIC RELATIVE EFFICIENCY

ABSTRACT

The purpose of this paper is to use Bahadur's asymptotic relative efficiency measure to compare the performance of various tests of autoregressive (AR) versus moving average (MA) error processes in regression models. Tests to be examined include non-nested procedures of the models against each other, and classical procedures based upon testing both the AR and MA error processes against the more general autoregressive-moving average model.     

Estimation Bias in the First-Order Autoregressive Model and Its Impact on Predictions and Prediction Intervals

The least squares estimate of the autoregressive coefficient in the AR(1) model is known to be biased towards zero, especially for parameters close to the stationarity boundary. Several methods for correcting the autoregressive parameter estimate for the bias have been suggested. Using simulations, we study the bias and the mean square error of the least squares estimate and the bias-corrections proposed by Kendall and Quenouille.

We also study the mean square forecast error and the coverage of the 95% prediction interval when using the biased least squares estimate or one of its bias-corrected versions. We find that the estimation bias matters little for point forecasts, but that it affects the coverage of the prediction intervals. Prediction intervals for forecasts more than one step ahead, when calculated with the biased least squares estimate, are too narrow.     

A cusum test in the linear regression model with serially correlated disturbances

This paper considers a modified CUSUM test, suggested by Dufour (1982) for parameter instability and structural change with an unknown change point in a linear model with serially correlated disturbances, in which a preliminary estimate of the autoregressive coefficient for the error process is obtained, and used to transform the data. Then the standard CUSUM statistic is calculated on the transformed data. This paper derives the asymptotic distribution of the modified CUSUM test. We show that the modified CUSUM test retains its asymptotic significance level, i.e., the modified CUSUM test has the same asymptotic distribution as the CUSUM test with serially uncorrelated errors.     

Goodness-of-fit tests for binomial AR(1) processes

The binomial AR(1) model describes a nonlinear process with a first-order autoregressive (AR(1)) structure and a binomial marginal distribution. To develop goodness-of-fit tests for the binomial AR(1) model, we investigate the observed marginal distribution of the binomial AR(1) process, and we tackle its autocorrelation structure. Motivated by the family of power-divergence statistics for handling discrete multivariate data, we derive the asymptotic distribution of certain categorized power-divergence statistics for the case of a binomial AR(1) process. Then we consider Bartlett's formula, which is widely used in time series analysis to provide estimates of the asymptotic covariance between sample autocorrelations, but which is not applicable when the underlying process is nonlinear. Hence, we derive a novel Bartlett-type formula for the asymptotic distribution of the sample autocorrelations of a binomial AR(1) process, which is then applied to develop tests concerning the autocorrelation structure. Simulation studies are carried out to evaluate the size and power of the proposed tests under diverse alternative process models. Several real examples are used to illustrate our methods and findings.