M‐estimation for general ARMA Processes with Infinite Variance

Abstract. General autoregressive moving average (ARMA) models extend the traditional ARMA models by removing the assumptions of causality and invertibility. The assumptions are not required under a non‐Gaussian setting for the identifiability of the model parameters in contrast to the Gaussian setting. We study M‐estimation for general ARMA processes with infinite variance, where the distribution of innovations is in the domain of attraction of a non‐Gaussian stable law. Following the approach taken by Davis et al. (1992) and Davis (1996) , we derive a functional limit theorem for random processes based on the objective function, and establish asymptotic properties of the M‐estimator. We also consider bootstrapping the M‐estimator and extend the results of Davis & Wu (1997) to the present setting so that statistical inferences are readily implemented. Simulation studies are conducted to evaluate the finite sample performance of the M‐estimation and bootstrap procedures. An empirical example of financial time series is also provided.     

Comparisons of coherent systems with non-identically distributed components

The signature-based mixture representations for coherent systems are a good way to obtain distribution-free comparisons of systems. Unfortunately, these representations only hold for systems whose component lifetimes are independent and identically distributed (IID) or exchangeable (i.e., their joint distribution is invariant under permutations). In this paper we obtain comparison results for generalized mixtures, that is, for reliability functions that can be written as linear combinations of some baseline reliability functions with positive and negative coefficients. These results are based on some concepts in Graph Theory. We apply these results to obtain new comparison results for coherent systems without the IID or exchangeability assumptions by using their generalized mixture representations based on the minimal path sets.     

Prequential omnibus goodness-of-fit tests for stochastic processes: A numerical study

This article is a contribution to the study of an omnibus goodness-of-fit (Gof) test based on Rosenblatt Probability Integral Transform (RPIT) within Dawid's prequential framework. This Gof test is easy to use since it has a common test statistic (with apparently the same asymptotic distribution) for a wide range of stochastic models. Intensive Monte-Carlo simulations are presented to investigate the behavior of this test for several stochastic models: renewal, autoregressive (AR, ARMA, ARCH, GARCH) and Poisson processes, generalized linear models... These simulations suggest that the RPIT test could be used to test the fit of a wide range of stochastic models but it may be not powerful when compared to Gof tests specifically designed for the tested processes. It is also conjectured that this test is still appropriate for testing the Gof of any discrete-time stochastic process provided that efficient estimators are used.     

The asymptotic behaviour of a class of L.-estimators under long-range dependence

This paper obtains asymptotic representations of a class of L-estimators in a linear regression model when the errors are a function of long-range-dependent Gaussian random variables. These representations are then used to address some of the efficiency robustness properties of L-estimators compared to the least-squares estimator. It is observed that under the Gaussian error distribution, each member of the class has the same asymptotic efficiency as that of the least-squares estimator. The results are obtained as a consequence of the asymptotic uniform linearity of some weighted empirical processes based on long-range-dependent random variables.     

Test for Parameter Change in Linear Processes Based on Whittle's Estimator

In this article, we develop a cusum test for testing for parameter changes in linear processes based on Whittle's estimator. It is shown that under regularity conditions, the test statistic converges to the sup of a Brownian bridge. The result is particularly useful in handling the change point test in stationary ARMA processes. A simulation result is provided for illustration.     

Unit root tests: the autoregressive approach in comparison with the periodogram regression

In Monte Carlo sudies we investigate unit root tests in line with Dickey/Fuller (1979). In case of positively autocorrelated MA(1) residuals their experimental power is extremely poor. Next we compare different versions of periodogram regression suggested in the literature. Their experimental behaviour is investigated with fractionally integrated processes. It is demonstrated how unit root tests may be based on periodogram regression. There is simulation evidence that those tests may do better in terms of power than the autoregressive tests, especially when testing ARMA(1,1) series against a linear time trend.     

Linear-representation Based Estimation of Stochastic Volatility Models

Abstract.  A new way of estimating stochastic volatility models is developed. The method is based on the existence of autoregressive moving average (ARMA) representations for powers of the log-squared observations. These representations allow to build a criterion obtained by weighting the sums of squared innovations corresponding to the different ARMA models. The estimator obtained by minimizing the criterion with respect to the parameters of interest is shown to be consistent and asymptotically normal. Monte-Carlo experiments illustrate the finite sample properties of the estimator. The method has potential applications to other non-linear time-series models.     

Probabilistic Properties of Parametric Dual and Inverse Time Series Models Generated by ARMA Models

For the class of autoregressive-moving average (ARMA) processes, we examine the relationship between the dual and the inverse processes. It is demonstrated that the inverse process generated by a causal and invertible ARMA (p, q) process is a causal and invertible ARMA (q, p) model. Moreover, it is established that this representation is strong if and only if the generating process is Gaussian. More precisely, it is derived that the linear innovation process of the inverse process is an all-pass model. Some examples and applications to time reversibility are given to illustrate the obtained results.     

Discrimination of AR,MA and ARMA time series models

The problem of discrimination between two stationary ARMA time series models is considered, and in particular AR(p), MA(p), ARMA(1,1) models. The discriminant based on the likelihood ration leads to a quadratic form that is generally too complicated to evaluated explicitly. The discriminant can be expressed approximately as a linear combination of independent chi–squared random varianles each with one degree of freedom, the coefficients, of which are eigenvalues of cumbersome matrices. An analytical solution which gives the coefficients approximately is suggested.     

On the identification of ARMA echelon-form models

An identification procedure for multivariate autoregressive moving average (ARMA) echelon-form models is proposed. It is based on the study of the linear dependence between rows of the Hankel matrix of serial correlations. To that end, we define a statistical test for checking the linear dependence between vectors of serial correlations. It is shown that the test statistic t?_n considered is distributed asymptotically as a finite linear combination of independent chi-square random variables with one degree of freedom under the null hypothesis, whereas under the alternative hypothesis, t?_N/N converges in probability to a positive constant. These results allow us, in particular, to compute the asymptotic probability of making a specification error with the proposed procedure. Links to other methods based on the application of canonical analysis are discussed. A simulation experiment was done in order to study the performance of the procedure. It is seen that the graphical representation of t?_N, as a function of N, can be very useful in identifying the dynamic structure of ARMA models. Furthermore, for the model considered, the proposed identification procedure performs very well for series of 100 observations or more and reasonably well with short series of 50 observations.     

Autoregressive moving-average convolution processes on the unit circle

ARMA convolution models for processes in continuous space (in this case the unit circle) and discrete time are derived as a natural extension of the usual Box-Jenkins models. Both weakly time-stationary and nonstationary processes are considered. Sufficient conditions for the existence of weakly time-stationary AR_cMA_c processes are derived, and the covariance functions for some processes are computed. It is demonstrated that the usual scalar and multivariate ARMA processes can be embedded within the larger class of AR_CMA_c models. A possible application of these models to sea-surface temperature prediction is discussed.     

Asymptotic Inefficiency of Mean-Correction on Parameter Estimation for a Periodic First-Order Autoregressive Model

A common practice in time series analysis is to fit a centered model to the mean-corrected data set. For stationary autoregressive moving-average (ARMA) processes, as far as the parameter estimation is concerned, fitting an ARMA model without intercepts to the mean-corrected series is asymptotically equivalent to fitting an ARMA model with intercepts to the observed series. We show that, related to the parameter least squares estimation of periodic ARMA models, the second approach can be arbitrarily more efficient than the mean-corrected counterpart. This property is illustrated by means of a periodic first-order autoregressive model. The asymptotic variance of the estimators for both approaches is derived. Moreover, empirical experiments based on simulations investigate the finite sample properties of the estimators.     

Bootstrap estimates of the sample bivariate autocorrelation and partial autocorrelation distributions

This paper shows how the bootstrap method can be used to estimate the joint distribution of sample autocorrelations and partial autocorrelations. The exact joint distribution of sample autocorrelations is mathematically intractable and attempts at workable approximations are difficult and rely on special assumptions. The bootstrap offers an accurate solution to this problem without requiring special assumptions and in a way that avoids theoretical difficulties. The bootstrap-estimated joint distributions of the autocorrelations and partial autocorrelations of time series are shown to lead to better ARMA model identification. This is demonstrated using simulated series.     

Some weighted mixed portmanteau tests for diagnostic checking in linear time series models

The popular diagnostic checking methods in linear time series models are portmanteau tests based on either residual autocorrelation functions (acf) or partial autocorrelation functions (pacf). In this paper, we device some new weighted mixed portmanteau tests by appropriately combining individual tests based on both acf and pacf. We derive the asymptotic distribution of such weighted mixed portmanteau statistics and study their size and power. It is found that the weighted mixed tests outperform when higher order ARMA models are fitted and diagnostic checks are performed via testing lack of residual autocorrelations. Simulation results suggest to use the proposed tests as complementary to those classical tests found in literature. An illustrative application is given to demonstrate the usefulness of the mixed test.     

Comparing aggregate and disaggregate forecasts of first order moving average models

This paper compares the performance of “aggregate” and “disaggregate” predictors in forecasting contemporaneously aggregated vector MA(1) processes. The necessary and sufficient condition for the equality of mean squared errors associated with the two competing predictors is provided in the bivariate MA(1) case. Furthermore, it is argued that the condition of equality of predictors as stated by Lütkepohl (Forecasting aggregated vector ARMA processes, Springer, Berlin, 1987) is only sufficient (not necessary) for the equality of mean squared errors. Finally, it is shown that the equality of forecasting accuracy for the two predictors can be achieved using specific assumptions on the parameters of the vector MA(1) structure.     

Testing lack of symmetry in spatial–temporal processes

Symmetry and separability of a covariance function are common assumptions to simplify the modeling effort of spatial–temporal processes. However, many studies in environmental sciences show that real data have complex spatial–temporal dependency structures resulting from lack of symmetry or violation of other standard assumptions of the covariance function. In this study, we propose new formal tests for lack of symmetry by using spectral representations of the spatial–temporal covariance functions of regularly spaced spatial–temporal data. The advantage of the proposed tests is that classical analysis of variance (ANOVA) models can be used for detecting lack of symmetry inherent in spatial–temporal processes. We evaluate the performance of the tests with simulation studies and we apply them to air pollution data.     

Time series models with asymmetric Laplace innovations

We propose autoregressive moving average (ARMA) and generalized autoregressive conditional heteroscedastic (GARCH) models driven by asymmetric Laplace (AL) noise. The AL distribution plays, in the geometric-stable class, the analogous role played by the normal in the alpha-stable class, and has shown promise in the modelling of certain types of financial and engineering data. In the case of an ARMA model we derive the marginal distribution of the process, as well as its bivariate distribution when separated by a finite number of lags. The calculation of exact confidence bands for minimum mean-squared error linear predictors is shown to be straightforward. Conditional maximum likelihood-based inference is advocated, and corresponding asymptotic results are discussed. The models are particularly suited for processes that are skewed, peaked, and leptokurtic, but which appear to have some higher order moments. A case study of a fund of real estate returns reveals that AL noise models tend to deliver a superior fit with substantially less parameters than normal noise counterparts, and provide both a competitive fit and a greater degree of numerical stability with respect to other skewed distributions.     

The Extensively Corrected Score for Measurement Error Models

In measurement error problems, two major and consistent estimation methods are the conditional score and the corrected score. They are functional methods that require no parametric assumptions on mismeasured covariates. The conditional score requires that a suitable sufficient statistic for the mismeasured covariate can be found, while the corrected score requires that the object score function can be estimated without bias. These assumptions limit their ranges of applications. The extensively corrected score proposed here is an extension of the corrected score. It yields consistent estimations in many cases when neither the conditional score nor the corrected score is feasible. We demonstrate its constructions in generalized linear models and the Cox proportional hazards model, assess its performances by simulation studies and illustrate its implementations by two real examples.     

Uncorrelated residuals and an exact test for two variance components in experimental design

The error contrasts from an experimental design can be constructed from uncorrelated residuals normally associated with the linear model. In this paper uncorrelated residuals are defined for the linear model that has a design matrix which is less than full rank, typical of many experimental design representations. It transpires in this setting, that for certain choices of uncorrelated residuals, corresponding to recursive type residuals, there is a natural partition of information when two variance components are known to be present. Under an assumtion of normality of errors this leads to construction of appropriate F-tests for testing heteroscedasticity. The test, which can be optimal, is applied to two well known data sets to illustrate its usefullness.     

Time series analysis of categorical data using auto-mutual information

Despite its importance, there has been little attention in the modeling of time series data of categorical nature in the recent past. In this paper, we present a framework based on the Pegram's [An autoregressive model for multilag Markov chains. Journal of Applied Probabability 17, 350–362] operator that was originally proposed only to construct discrete AR(p$p$) processes. We extend the Pegram's operator to accommodate categorical processes with ARMA representations. We observe that the concept of correlation is not always suitable for categorical data. As a sensible alternative, we use the concept of mutual information, and introduce auto-mutual information to define the time series process of categorical data. Some model selection and inferential aspects are also discussed. We implement the developed methodologies to analyze a time series data set on infant sleep status.