PREDICTION-FOCUSED MODEL SELECTION FOR AUTOREGRESSIVE MODELS

In order to make predictions of future values of a time series, one needs to specify a forecasting model. A popular choice is an autoregressive time‐series model, for which the order of the model is chosen by an information criterion. We propose an extension of the focused information criterion (FIC) for model‐order selection, with emphasis on a high predictive accuracy (i.e. the mean squared forecast error is low). We obtain theoretical results and illustrate by means of a simulation study and some real data examples that the FIC is a valid alternative to the Akaike information criterion (AIC) and the Bayesian information criterion (BIC) for selection of a prediction model. We also illustrate the possibility of using the FIC for purposes other than forecasting, and explore its use in an extended model.     

MODEL SELECTION FOR PENALIZED SPLINE SMOOTHING USING AKAIKE INFORMATION CRITERIA

Two different forms of Akaike's information criterion (AIC) are compared for selecting the smooth terms in penalized spline additive mixed models. The conditional AIC (cAIC) has been used traditionally as a criterion for both estimating penalty parameters and selecting covariates in smoothing, and is based on the conditional likelihood given the smooth mean and on the effective degrees of freedom for a model fit. By comparison, the marginal AIC (mAIC) is based on the marginal likelihood from the mixed‐model formulation of penalized splines which has recently become popular for estimating smoothing parameters. To the best of the authors' knowledge, the use of mAIC for selecting covariates for smoothing in additive models is new. In the competing models considered for selection, covariates may have a nonlinear effect on the response, with the possibility of group‐specific curves. Simulations are used to compare the performance of cAIC and mAIC in model selection settings that have correlated and hierarchical smooth terms. In moderately large samples, both formulations of AIC perform extremely well at detecting the function that generated the data. The mAIC does better for simple functions, whereas the cAIC is more sensitive to detecting a true model that has complex and hierarchical terms.     

Discrimination of AR,MA and ARMA time series models

The problem of discrimination between two stationary ARMA time series models is considered, and in particular AR(p), MA(p), ARMA(1,1) models. The discriminant based on the likelihood ration leads to a quadratic form that is generally too complicated to evaluated explicitly. The discriminant can be expressed approximately as a linear combination of independent chi–squared random varianles each with one degree of freedom, the coefficients, of which are eigenvalues of cumbersome matrices. An analytical solution which gives the coefficients approximately is suggested.     

An Application of Vector Time Series Techniques to Macroeconomic Forecasting

A wide variety of time series techniques are now used for generating forecasts of economic variables, with each technique attempting to summarize and exploit whatever regularities exist in a given data set. It appears that many researchers arbitrarily choose one of these techniques. The purpose of this article is to provide an example for which the choice of time series technique appears important; merely choosing arbitrarily among available techniques may lead to suboptimal results.     

A SIMPLE VARIABLE SELECTION TECHNIQUE FOR NONLINEAR MODELS

Applying nonparametric variable selection criteria in nonlinear regression models generally requires a substantial computational effort if the data set is large. In this paper we present a selection technique that is computationally much less demanding and performs well in comparison with methods currently available. It is based on a polynomial approximation of the nonlinear model. Performing the selection only requires repeated least squares estimation of models that are linear in parameters. The main limitation of the method is that the number of variables among which to select cannot be very large if the sample is small and the order of an adequate polynomial at the same time is high. Large samples can be handled without problems.     

Effect of systematic sampling on arima models

Given a general homogeneous non-stationary autoregressive integrated moving average process ARIMA(p,d,q), the corresponding model for the subseries obtained by a systematic sampling is derived. The article then shows that the sampled subseries approaches approximately to an integrated moving average process IMA(d,l), l≤(d-l), regardless of the autoregressive and moving average structures in the original series. In particular, the sampled subseries from an ARIMA (p,l,q) process approaches approximately to a simple random walk model.     

Multivariate time series prediction using a hybridization of VARMA models and Bayesian networks

In this paper, a new hybrid model of vector autoregressive moving average (VARMA) models and Bayesian networks is proposed to improve the forecasting performance of multivariate time series. In the proposed model, the VARMA model, which is a popular linear model in time series forecasting, is specified to capture the linear characteristics. Then the errors of the VARMA model are clustered into some trends by K-means algorithm with Krzanowski–Lai cluster validity index determining the number of trends, and a Bayesian network is built to learn the relationship between the data and the trend of its corresponding VARMA error. Finally, the estimated values of the VARMA model are compensated by the probabilities of their corresponding VARMA errors belonging to each trend, which are obtained from the Bayesian network. Compared with VARMA models, the experimental results with a simulation study and two multivariate real-world data sets indicate that the proposed model can effectively improve the prediction performance.     

ESTIMATION FOR THE GENERAL SAMPLE SELECTION MODELS

Consider a general regression model with an arbitrary and unknown link function and a stochastic selection variable that determines whether the outcome variable is observable or missing. The paper proposes U-statistics that are based on kernel functions as estimators for the directions of the parameter vectors in the link function and the selection equation, and shows that these estimators are consistent and asymptotically normal.     

ON A TEST OF FIT IN TIME SERIES MODELS

It is shown that a test for adequacy of a fitted Arma (p, 4) model based on forecast errors, has the same asymptotic expected value whether the null hypothesis is true or false. A sampling study supports the conclusion in small to moderately large samples and indicates that for sample sizes commonly used by practitioners, the power of the test is very low.     

ARMA MODELS REALIZATION AND IMPULSE RESPONSES

A parallel is made between the role played by covariances in the determination of auto-regressive models and the role played by impulse responses in the determination of ARMA models.

Auto-regressive models are known to maximize the Burg-entropy under covariance constraints. Auto-regressive-moving-average models give the maximum of the Burg-entropy among processes sharing the same covariances and impulse responses up to a certain lag. Such models are constructed by iterative or algebraic methods under the different constraints.

A new recursive method of identification of the order of an ARMA model is also developed, based on the generalized reflection coefficients.     

ON THE CAUSALITY TEST IN TIME SERIES MODELS WITH HEAVY-TAILED DISTRIBUTION

ABSTRACT

In this article, we consider the problem of testing the Granger causality in stationary time series models with non-normal heavy-tailed distributions. We consider a normal mixture model to cover the heavy-tailed distribution, and propose a test statistic based on the partially adaptive estimator proposed by Phillips [1] Phillips, R.F. 1994. Partially Adaptive Estimation via a Normal Mixture. J. Econometics, 64: 123–144. [Crossref], [Web of Science ®] , [Google Scholar]. It is shown that the test statistic asymptotically follows a chi-squared distribution. Simulation results indicate that our test outperforms the conventional test based on the least squares estimator when the observations follow a heavy-tailed distribution.     

A METHOD FOR ESTIMATING PARAMETER IN NONNEGATIVE MA(1) MODELS

ABSTRACT

A method for estimating parameter in nonnegative MA(1) models is proposed and investigated in the paper. The method also gives nontrivial confidence sets on confidence level 1. Small sample properties of new estimator are demonstrated in a simulation study.     

Consistent testing for non‐correlation of two cointegrated ARMA time series

A consistent approach to the problem of testing non‐correlation between two univariate infinite‐order autoregressive models was proposed by Hong (1996). His test is based on a weighted sum of squares of residual cross‐correlations, with weights depending on a kernel function. In this paper, the author follows Hong's approach to test non‐correlation of two cointegrated (or partially non‐stationary) ARMA time series. The test of Pham, Roy & Cédras (2003) may be seen as a special case of his approach, as it corresponds to the choice of a truncated uniform kernel. The proposed procedure remains valid for testing non‐correlation between two stationary invertible multivariate ARMA time series. The author derives the asymptotic distribution of his test statistics under the null hypothesis and proves that his procedures are consistent. He also studies the level and power of his proposed tests in finite samples through simulation. Finally, he presents an illustration based on real data.     

The effects of different choices of order for autoregressive approximation on the Gaussian likelihood estimates for ARMA models

This paper investigates, by means of Monte Carlo simulation, the effects of different choices of order for autoregressive approximation on the fully efficient parameter estimates for autoregressive moving average models. Four order selection criteria, AIC, BIC, HQ and PKK, were compared and different model structures with varying sample sizes were used to contrast the performance of the criteria. Some asymptotic results which provide a useful guide for assessing the performance of these criteria are presented. The results of this comparison show that there are marked differences in the accuracy implied using these alternative criteria in small sample situations and that it is preferable to apply BIC criterion, which leads to greater precision of Gaussian likelihood estimates, in such cases. Implications of the findings of this study for the estimation of time series models are highlighted.     

A simulation study of l1:estimation of a seasonal moving average time series model

A detailed simulation study is reported on the application of l₁:estimations to a seasonal moving average model. It is found that the asymptotic normal distribution is a nonapproximation to the finite sample distribution. However, the expected benefits of l₁:estimation relative to l₂:are partially realised for nonnormal innovative distributions.     

Focused information criterion for locally misspecified vector autoregressive models

This paper investigates the focused information criterion and plug-in average for vector autoregressive models with local-to-zero misspecification. These methods have the advantage of focusing on a quantity of interest rather than aiming at overall model fit. Any (su?ciently regular) function of the parameters can be used as a quantity of interest. We determine the asymptotic properties and elaborate on the role of the locally misspecified parameters. In particular, we show that the inability to consistently estimate locally misspecified parameters translates into suboptimal selection and averaging. We apply this framework to impulse response analysis. A Monte Carlo simulation study supports our claims.     

Trends and Cycles in Macroeconomic Time Series

Two structural time series models for annual observations are constructed in terms of trend, cycle, and irregular components. The models are then estimated via the Kalman filter using data on five U.S. macroeconomic time series. The results provide some interesting insights into the dynamic structure of the series, particularly with respect to cyclical behavior. At the same time, they illustrate the development of a model selection strategy for structural time series models.     

SOME SECOND ORDER ASYMPTOTICS IN EXPONENTIAL FAMILY NON-LINEAR REGRESSION MODELS (A GEOMETRIC APPROACH)

The differential geometric framework of Amari (1982a, 1985) is applied to the study of some second order asymptotics related to the curvatures for exponential family nonlinear regression models, in which the observations are independent but not necessarily identically distributed. This paper presents a set of reasonable regularity conditions which are needed to study asymptotics from a geometric point of view in regression models. A new stochastic expansion of a first order efficient estimator is derived and used to study several asymptotic problems related to Fisher information in terms of curvatures. The bias and the covariance of the first order efficient estimator are also calculated according to the expansion.     

Some Properties and Generalizations of Non-negative Bayesian Time Series Models

We study the most basic Bayesian forecasting model for exponential family time series, the power steady model (PSM) of Smith, in terms of observable properties of one-step forecast distributions and sample paths. The PSM implies a constraint between location and spread of the forecast distribution. Including a scale parameter in the models does not always give an exact solution free of this problem, but it does suggest how to define related models free of the constraint. We define such a class of models which contains the PSM. We concentrate on the case where observations are non-negative. Probability theory and simulation show that under very mild conditions almost all sample paths of these models converge to some constant, making them unsuitable for modelling in many situations. The results apply more generally to non-negative models defined in terms of exponentially weighted moving averages. We use these and related results to motivate, define and apply very simple models based on directly specifying the forecast distributions.     

LOSS OF INFORMATION ASSOCIATED WITH THE ORDER STATISTICS AND RELATED ESTIMATORS IN THE DOUBLE EXPONENTIAL DISTRIBUTION CASE

Fisher (1934) derived the loss of information of the maximum likelihood estimator (MLE) of the location parameter in the case of the double exponential distribution. Takeuchi & Akahira (1976) showed that the MLE is not second order asymptotically efficient. This paper extends these results by obtaining the (asymptotic) losses of information of order statistics and related estimators, and by comparing them via their asymptotic distributions up to the second order.