Bayesian classification with multivariate autoregressive sources that might have different orders

The main objective of this paper is to develop an exact Bayesian technique that can be used to assign a multivariate time series realization to one of several autoregressive sources, with unknown coefficients and precision, that might have different orders. The foundation of the proposed technique is to develop the posterior mass function of a classification vector, in an easy form, using the conditional likelihood function. A multivariate time series realization is assigned to the multivariate autoregressive source with the largest posterior probability. A simulation study, with uniform prior mass function, is carried out to demonstrate the performance of the proposed technique and to test its adequacy in handling the multivariate classification problems. The analysis of the numerical results supports the adequacy of the proposed technique in solving the classification problems with multivariate autoregressive sources.     

Bayesian Identification of Seasonal Autoregressive Models

Abstract

A very important and essential phase of time series analysis is identifying the model orders. This article develops an approximate Bayesian procedure to identify the orders of seasonal autoregressive processes. Using either a normal-gamma prior density or a noninformative prior, which is combined with an approximate conditional likelihood function, the foundation of the proposed technique is to derive the joint posterior mass function of the model orders in an easy form. Then one may inspect the posterior mass function and choose the orders with the largest posterior probability to be the suitable orders of the time series being analyzed. A simulation study, with different priors mass functions, is carried out to test the adequacy of the proposed technique and compare it with some non-Bayesian automatic criteria. The analysis of the numerical results supports the adequacy of the proposed technique in identifying the orders of the autoregressive processes.     

Bayesian Identification of Moving Average Models

This study approaches the Bayesian identification of moving average processes using an approximate likelihood function and a normal gamma prior density. The marginal posterior probability mass function of the model order is developed in a convenient form. Then one may investigate the posterior probabilities over the grid of the order and choose the order with the highest probability to solve the identification problem. A comprehensive simulation study is carried out to demonstrate the performance of the proposed procedure and check its adequacy in handling the identification problem. In addition, the proposed Bayesian procedure is compared with some non Bayesian automatic techniques and another Bayesian technique. The numerical results support the adequacy of using the proposed procedure in solving the identification problem of moving average processes.     

Structured priors for multivariate time series

A class of prior distributions for multivariate autoregressive models is presented. This class of priors is built taking into account the latent component structure that characterizes a collection of autoregressive processes. In particular, the state-space representation of a vector autoregressive process leads to the decomposition of each time series in the multivariate process into simple underlying components. These components may have a common structure across the series. A key feature of the proposed priors is that they allow the modeling of such common structure. This approach also takes into account the uncertainty in the number of latent processes, consequently handling model order uncertainty in the multivariate autoregressive framework. Posterior inference is achieved via standard Markov chain Monte Carlo (MCMC) methods. Issues related to inference and exploration of the posterior distribution are discussed. We illustrate the methodology analyzing two data sets: a synthetic data set with quasi-periodic latent structure, and seasonally adjusted US monthly housing data consisting of housing starts and housing sales over the period 1965 to 1974.     

Bayesian identification of double seasonal autoregressive time series models

ABSTRACT

Seasonal autoregressive (SAR) models have been modified and extended to model high frequency time series characterized by exhibiting double seasonal patterns. Some researchers have introduced Bayesian inference for double seasonal autoregressive (DSAR) models; however, none has tackled the problem of Bayesian identification of DSAR models. Therefore, in order to fill this gap, we present a Bayesian methodology to identify the order of DSAR models. Assuming the model errors are normally distributed and using three priors, i.e. natural conjugate, g, and Jeffreys’ priors, on the model parameters, we derive the joint posterior mass function of the model order in a closed-form. Accordingly, the posterior mass function can be investigated and the best order of DSAR model is chosen as a value with the highest posterior probability for the time series being analyzed. We evaluate the proposed Bayesian methodology using simulation study, and we then apply it to real-world hourly internet amount of traffic dataset.     

Priors and component structures in autoregressive time series models

New approaches to prior specification and structuring in autoregressive time series models are introduced and developed. We focus on defining classes of prior distributions for parameters and latent variables related to latent components of an autoregressive model for an observed time series. These new priors naturally permit the incorporation of both qualitative and quantitative prior information about the number and relative importance of physically meaningful components that represent low frequency trends, quasi-periodic subprocesses and high frequency residual noise components of observed series. The class of priors also naturally incorporates uncertainty about model order and hence leads in posterior analysis to model order assessment and resulting posterior and predictive inferences that incorporate full uncertainties about model order as well as model parameters. Analysis also formally incorporates uncertainty and leads to inferences about unknown initial values of the time series, as it does for predictions of future values. Posterior analysis involves easily implemented iterative simulation methods, developed and described here. One motivating field of application is climatology, where the evaluation of latent structure, especially quasi-periodic structure, is of critical importance in connection with issues of global climatic variability. We explore the analysis of data from the southern oscillation index, one of several series that has been central in recent high profile debates in the atmospheric sciences about recent apparent trends in climatic indicators.     

Bayesian Inferences and Forecasts With Multiple Autoregressive Moving Average Models

The main objective of this paper is to develop convenient Bayesian techniques for estimation and forecasting which can be used to analyze multiple (multivariate) autoregressive moving average processes. Based on the conditional likelihood function and the least squares estimates of the residuals, the marginal posterior distribution of the coefficients of the model is approximated by a matrix t distribution, the marginal posterior distribution of the precision matrix is approximated by a Wishart distribution, and the predictive distribution is approximated by a multivariate t distribution. Some numerical examples are given to demonstrate the idea of using the proposed techniques to analyze different types of multiple ARMA models.     

Bayesian analysis of multivariate threshold autoregressive models with missing data

In some fields, we are forced to work with missing data in multivariate time series. Unfortunately, the data analysis in this context cannot be carried out in the same way as in the case of complete data. To deal with this problem, a Bayesian analysis of multivariate threshold autoregressive models with exogenous inputs and missing data is carried out. In this paper, Markov chain Monte Carlo methods are used to obtain samples from the involved posterior distributions, including threshold values and missing data. In order to identify autoregressive orders, we adapt the Bayesian variable selection method in this class of multivariate process. The number of regimes is estimated using marginal likelihood or product parameter-space strategies.     

Cross-entropy method for estimation of posterior expectation in Bayesian VAR models

In this article, an importance sampling (IS) method for the posterior expectation of a non linear function in a Bayesian vector autoregressive (VAR) model is developed. Most Bayesian inference problems involve the evaluation of the expectation of a function of interest, usually a non linear function of the model parameters, under the posterior distribution. Non linear functions in Bayesian VAR setting are difficult to estimate and usually require numerical methods for their evaluation. A weighted IS estimator is used for the evaluation of the posterior expectation. With the cross-entropy (CE) approach, the IS density is chosen from a specified family of densities such that the CE distance or the Kullback–Leibler divergence between the optimal IS density and the importance density is minimal. The performance of the proposed algorithm is assessed in an iterated multistep forecasting of US macroeconomic time series.     

Dependence Estimation for High‐frequency Sampled Multivariate CARMA Models

The paper considers high‐frequency sampled multivariate continuous‐time autoregressive moving average (MCARMA) models and derives the asymptotic behaviour of the sample autocovariance function to a normal random matrix. Moreover, we obtain the asymptotic behaviour of the cross‐covariances between different components of the model. We will see that the limit distribution of the sample autocovariance function has a similar structure in the continuous‐time and in the discrete‐time model. As a special case, we consider a CARMA (one‐dimensional MCARMA) process. For a CARMA process, we prove Bartlett's formula for the sample autocorrelation function. Bartlett's formula has the same form in both models; only the sums in the discrete‐time model are exchanged by integrals in the continuous‐time model. Finally, we present limit results for multivariate MA processes as well, which are not known in this generality in the multivariate setting yet.     

Statistical Analysis Of Mixture Vector Autoregressive Models

In this paper, we reconsider the mixture vector autoregressive model, which was proposed in the literature for modelling non‐linear time series. We complete and extend the stationarity conditions, derive a matrix formula in closed form for the autocovariance function of the process and prove a result on stable vector autoregressive moving‐average representations of mixture vector autoregressive models. For these results, we apply techniques related to a Markovian representation of vector autoregressive moving‐average processes. Furthermore, we analyse maximum likelihood estimation of model parameters by using the expectation–maximization algorithm and propose a new iterative algorithm for getting the maximum likelihood estimates. Finally, we study the model selection problem and testing procedures. Several examples, simulation experiments and an empirical application based on monthly financial returns illustrate the proposed procedures.     

Improved estimation for Poisson INAR(1) models

We consider the first-order Poisson autoregressive model proposed by McKenzie [Some simple models for discrete variate time series. Water Resour Bull. 1985;21:645–650] and Al-Osh and Alzaid [First-order integer valued autoregressive (INAR(1)) process. J Time Ser Anal. 1987;8:261–275], which may be suitable in situations where the time series data are non-negative and integer valued. We derive the second-order bias of the squared difference estimator [Weiß. Process capability analysis for serially dependent processes of Poisson counts. J Stat Comput Simul. 2012;82:383–404] for one of the parameters and show that this bias can be used to define a bias-reduced estimator. The behaviour of a modified conditional least-squares estimator is also studied. Furthermore, we access the asymptotic properties of the estimators here discussed. We present numerical evidence, based upon Monte Carlo simulation studies, showing that the here proposed bias-adjusted estimator outperforms the other estimators in small samples. We also present an application to a real data set.     

On the estimation problem of periodic autoregressive time series: symmetric and asymmetric innovations

Periodic autoregressive (PAR) models with symmetric innovations are widely used on time series analysis, whereas its asymmetric counterpart inference remains a challenge, because of a number of problems related to the existing computational methods. In this paper, we use an interesting relationship between periodic autoregressive and vector autoregressive (VAR) models to study maximum likelihood and Bayesian approaches to the inference of a PAR model with normal and skew-normal innovations, where different kinds of estimation methods for the unknown parameters are examined. Several technical difficulties which are usually complicated to handle are reported. Results are compared with the existing classical solutions and the practical implementations of the proposed algorithms are illustrated via comprehensive simulation studies. The methods developed in the study are applied and illustrate a real-time series. The Bayes factor is also used to compare the multivariate normal model versus the multivariate skew-normal model.     

A bayesian analysis of trend determination in economic time series

In this paper we provide a comprehensive Bayesian posterior analysis of trend determination in general autoregressive models. Multiple lag autoregressive models with fitted drifts and time trends as well as models that allow for certain types of structural change in the deterministic components are considered. We utilize a modified information matrix-based prior that accommodates stochastic nonstationarity, takes into account the interactions between long-run and short-run dynamics and controls the degree of stochastic nonstationarity permitted. We derive analytic posterior densities for all of the trend determining parameters via the Laplace approximation to multivariate integrals. We also address the sampling properties of our posteriors under alternative data generating processes by simulation methods. We apply our Bayesian techniques to the Nelson-Plosser macroeconomic data and various stock price and dividend data. Contrary to DeJong and Whiteman (1989a,b,c), we do not find that the data overwhelmingly favor the existence of deterministic trends over stochastic trends. In addition, we find evidence supporting Perron's (1989) view that some of the Nelson and Plosser data are best construed as trend stationary with a change in the trend function occurring at 1929.     

A bayesian analysis of trend determination in economic time series

In this paper we provide a comprehensive Bayesian posterior analysis of trend determination in general autoregressive models. Multiple lag autoregressive models with fitted drifts and time trends as well as models that allow for certain types of structural change in the deterministic components are considered. We utilize a modified information matrix-based prior that accommodates stochastic nonstationarity, takes into account the interactions between long-run and short-run dynamics and controls the degree of stochastic nonstationarity permitted. We derive analytic posterior densities for all of the trend determining parameters via the Laplace approximation to multivariate integrals. We also address the sampling properties of our posteriors under alternative data generating processes by simulation methods. We apply our Bayesian techniques to the Nelson-Plosser macroeconomic data and various stock price and dividend data. Contrary to DeJong and Whiteman (1989a,b,c), we do not find that the data overwhelmingly favor the existence of deterministic trends over stochastic trends. In addition, we find evidence supporting Perron's (1989) view that some of the Nelson and Plosser data are best construed as trend stationary with a change in the trend function occurring at 1929.     

On a mixture vector autoregressive model

The authors show how to extend univariate mixture autoregressive models to a multivariate time series context. Similar to the univariate case, the multivariate model consists of a mixture of stationary or nonstationary autoregressive components. The authors give the first and second order stationarity conditions for a multivariate case up to order 2. They also derive the second order stationarity condition for the univariate mixture model up to arbitrary order. They describe an EM algorithm for estimation, as well as a diagnostic checking procedure. They study the performance of their method via simulations and include a real application.     

Analysis of Poisson varying-coefficient models with autoregression

In the regression analysis of time series of event counts, it is of interest to account for serial dependence that is likely to be present among such data as well as a nonlinear interaction between the expected event counts and predictors as a function of some underlying variables. We thus develop a Poisson autoregressive varying-coefficient model, which introduces autocorrelation through a latent process and allows regression coefficients to nonparametrically vary as a function of the underlying variables. The nonparametric functions for varying regression coefficients are estimated with data-driven basis selection, thereby avoiding overfitting and adapting to curvature variation. An efficient posterior sampling scheme is devised to analyse the proposed model. The proposed methodology is illustrated using simulated data and daily homicide data in Cali, Colombia.     

A semiparametric approach for modelling multivariate nonlinear time series

In this article, a semiparametric time‐varying nonlinear vector autoregressive (NVAR) model is proposed to model nonlinear vector time series data. We consider a combination of parametric and nonparametric estimation approaches to estimate the NVAR function for both independent and dependent errors. We use the multivariate Taylor series expansion of the link function up to the second order which has a parametric framework as a representation of the nonlinear vector regression function. After the unknown parameters are estimated by the maximum likelihood estimation procedure, the obtained NVAR function is adjusted by a nonparametric diagonal matrix, where the proposed adjusted matrix is estimated by the nonparametric kernel estimator. The asymptotic consistency properties of the proposed estimators are established. Simulation studies are conducted to evaluate the performance of the proposed semiparametric method. A real data example on short‐run interest rates and long‐run interest rates of United States Treasury securities is analyzed to demonstrate the application of the proposed approach. The Canadian Journal of Statistics 47: 668–687; 2019 © 2019 Statistical Society of Canada     

On hysteretic vector autoregressive model with applications

This paper proposes a new hysteretic vector autoregressive (HVAR) model in which the regime switching may be delayed when the hysteresis variable lies in a hysteresis zone. We integrate an adapted multivariate Student-t distribution from amending the scale mixtures of normal distributions. This HVAR model allows for a higher degree of flexibility in the degrees of freedom for each time series. We use the proposed model to test for a causal relationship between any two target time series. Using posterior odds ratios, we overcome the limitations of the classical approach to multiple testing. Both simulated and real examples herein help illustrate the suggested methods. We apply the proposed HVAR model to investigate the causal relationship between the quarterly growth rates of gross domestic product of United Kingdom and United States. Moreover, we check the pairwise lagged dependence of daily PM2.5 levels in three districts of Taipei.