Bayesian Identification of Multivariate Autoregressive Processes

Identification is one of the most important stages of a time series analysis. This paper develops a direct Bayesian technique to identify the order of multivariate autoregressive processes. By employing the conditional likelihood function and a matrix normal-Wishart prior density, or Jeffrey' vague prior, the proposed identification technique is based on deriving the exact posterior probability mass function of the model order in a convenient form. Then one may easily evaluate the posterior probabilities of the model order and choose the order that maximizes the posterior mass function to be the suitable order of the time series data being analyzed. Assuming the bivariate autoregressive processes, a numerical study, with different prior mass functions, is carried out to assess the efficiency of the proposed technique. The analysis of the numerical results supports the adequacy of the proposed technique in identifying the orders of multivariate autoregressive processes.     

Bayesian Identification of Seasonal Autoregressive Models

Abstract

A very important and essential phase of time series analysis is identifying the model orders. This article develops an approximate Bayesian procedure to identify the orders of seasonal autoregressive processes. Using either a normal-gamma prior density or a noninformative prior, which is combined with an approximate conditional likelihood function, the foundation of the proposed technique is to derive the joint posterior mass function of the model orders in an easy form. Then one may inspect the posterior mass function and choose the orders with the largest posterior probability to be the suitable orders of the time series being analyzed. A simulation study, with different priors mass functions, is carried out to test the adequacy of the proposed technique and compare it with some non-Bayesian automatic criteria. The analysis of the numerical results supports the adequacy of the proposed technique in identifying the orders of the autoregressive processes.     

Bayesian identification of double seasonal autoregressive time series models

ABSTRACT

Seasonal autoregressive (SAR) models have been modified and extended to model high frequency time series characterized by exhibiting double seasonal patterns. Some researchers have introduced Bayesian inference for double seasonal autoregressive (DSAR) models; however, none has tackled the problem of Bayesian identification of DSAR models. Therefore, in order to fill this gap, we present a Bayesian methodology to identify the order of DSAR models. Assuming the model errors are normally distributed and using three priors, i.e. natural conjugate, g, and Jeffreys’ priors, on the model parameters, we derive the joint posterior mass function of the model order in a closed-form. Accordingly, the posterior mass function can be investigated and the best order of DSAR model is chosen as a value with the highest posterior probability for the time series being analyzed. We evaluate the proposed Bayesian methodology using simulation study, and we then apply it to real-world hourly internet amount of traffic dataset.     

Bayesian Identification of Moving Average Models

This study approaches the Bayesian identification of moving average processes using an approximate likelihood function and a normal gamma prior density. The marginal posterior probability mass function of the model order is developed in a convenient form. Then one may investigate the posterior probabilities over the grid of the order and choose the order with the highest probability to solve the identification problem. A comprehensive simulation study is carried out to demonstrate the performance of the proposed procedure and check its adequacy in handling the identification problem. In addition, the proposed Bayesian procedure is compared with some non Bayesian automatic techniques and another Bayesian technique. The numerical results support the adequacy of using the proposed procedure in solving the identification problem of moving average processes.     

Structured priors for multivariate time series

A class of prior distributions for multivariate autoregressive models is presented. This class of priors is built taking into account the latent component structure that characterizes a collection of autoregressive processes. In particular, the state-space representation of a vector autoregressive process leads to the decomposition of each time series in the multivariate process into simple underlying components. These components may have a common structure across the series. A key feature of the proposed priors is that they allow the modeling of such common structure. This approach also takes into account the uncertainty in the number of latent processes, consequently handling model order uncertainty in the multivariate autoregressive framework. Posterior inference is achieved via standard Markov chain Monte Carlo (MCMC) methods. Issues related to inference and exploration of the posterior distribution are discussed. We illustrate the methodology analyzing two data sets: a synthetic data set with quasi-periodic latent structure, and seasonally adjusted US monthly housing data consisting of housing starts and housing sales over the period 1965 to 1974.     

Bayesian Inferences and Forecasts With Multiple Autoregressive Moving Average Models

The main objective of this paper is to develop convenient Bayesian techniques for estimation and forecasting which can be used to analyze multiple (multivariate) autoregressive moving average processes. Based on the conditional likelihood function and the least squares estimates of the residuals, the marginal posterior distribution of the coefficients of the model is approximated by a matrix t distribution, the marginal posterior distribution of the precision matrix is approximated by a Wishart distribution, and the predictive distribution is approximated by a multivariate t distribution. Some numerical examples are given to demonstrate the idea of using the proposed techniques to analyze different types of multiple ARMA models.     

Cross-entropy method for estimation of posterior expectation in Bayesian VAR models

In this article, an importance sampling (IS) method for the posterior expectation of a non linear function in a Bayesian vector autoregressive (VAR) model is developed. Most Bayesian inference problems involve the evaluation of the expectation of a function of interest, usually a non linear function of the model parameters, under the posterior distribution. Non linear functions in Bayesian VAR setting are difficult to estimate and usually require numerical methods for their evaluation. A weighted IS estimator is used for the evaluation of the posterior expectation. With the cross-entropy (CE) approach, the IS density is chosen from a specified family of densities such that the CE distance or the Kullback–Leibler divergence between the optimal IS density and the importance density is minimal. The performance of the proposed algorithm is assessed in an iterated multistep forecasting of US macroeconomic time series.     

On the estimation problem of periodic autoregressive time series: symmetric and asymmetric innovations

Periodic autoregressive (PAR) models with symmetric innovations are widely used on time series analysis, whereas its asymmetric counterpart inference remains a challenge, because of a number of problems related to the existing computational methods. In this paper, we use an interesting relationship between periodic autoregressive and vector autoregressive (VAR) models to study maximum likelihood and Bayesian approaches to the inference of a PAR model with normal and skew-normal innovations, where different kinds of estimation methods for the unknown parameters are examined. Several technical difficulties which are usually complicated to handle are reported. Results are compared with the existing classical solutions and the practical implementations of the proposed algorithms are illustrated via comprehensive simulation studies. The methods developed in the study are applied and illustrate a real-time series. The Bayes factor is also used to compare the multivariate normal model versus the multivariate skew-normal model.     

Analysis of Poisson varying-coefficient models with autoregression

In the regression analysis of time series of event counts, it is of interest to account for serial dependence that is likely to be present among such data as well as a nonlinear interaction between the expected event counts and predictors as a function of some underlying variables. We thus develop a Poisson autoregressive varying-coefficient model, which introduces autocorrelation through a latent process and allows regression coefficients to nonparametrically vary as a function of the underlying variables. The nonparametric functions for varying regression coefficients are estimated with data-driven basis selection, thereby avoiding overfitting and adapting to curvature variation. An efficient posterior sampling scheme is devised to analyse the proposed model. The proposed methodology is illustrated using simulated data and daily homicide data in Cali, Colombia.     

Stable Autoregressive Models and Signal Estimation

This article studies the problem of model identification and estimation for stable autoregressive process observed in a symmetric stable noise environment. A new tool called partial auto-covariation function is introduced to identify the stable autoregressive signals. The signal and noise parameters are estimated using a modified version of Generalized Yule Walker type method and the method of moments. The proposed methods are illustrated through data simulated from autoregressive signals with symmetric stable innovations. The new technique is applied to analyze the time series of sea surface temperature anomaly and compared with its Gaussian counterpart.     

Bayesian analysis of multivariate threshold autoregressive models with missing data

In some fields, we are forced to work with missing data in multivariate time series. Unfortunately, the data analysis in this context cannot be carried out in the same way as in the case of complete data. To deal with this problem, a Bayesian analysis of multivariate threshold autoregressive models with exogenous inputs and missing data is carried out. In this paper, Markov chain Monte Carlo methods are used to obtain samples from the involved posterior distributions, including threshold values and missing data. In order to identify autoregressive orders, we adapt the Bayesian variable selection method in this class of multivariate process. The number of regimes is estimated using marginal likelihood or product parameter-space strategies.     

Classification Rules under Autoregressive and General Circulant Covariance

We develop classification rules for data that have an autoregressive circulant covariance structure under the assumption of multivariate normality. We also develop classification rules assuming a general circulant covariance structure. The new classification rules are efficient in reducing the misclassification error rates when the number of observations is not large enough to estimate the unknown variance–covariance matrix. The proposed classification rules are demonstrated by simulation study for their validity and illustrated by a real data analysis for their use. Analyses of both simulated data and real data show the effectiveness of our new classification rules.     

Consistent testing for non‐correlation of two cointegrated ARMA time series

A consistent approach to the problem of testing non‐correlation between two univariate infinite‐order autoregressive models was proposed by Hong (1996). His test is based on a weighted sum of squares of residual cross‐correlations, with weights depending on a kernel function. In this paper, the author follows Hong's approach to test non‐correlation of two cointegrated (or partially non‐stationary) ARMA time series. The test of Pham, Roy & Cédras (2003) may be seen as a special case of his approach, as it corresponds to the choice of a truncated uniform kernel. The proposed procedure remains valid for testing non‐correlation between two stationary invertible multivariate ARMA time series. The author derives the asymptotic distribution of his test statistics under the null hypothesis and proves that his procedures are consistent. He also studies the level and power of his proposed tests in finite samples through simulation. Finally, he presents an illustration based on real data.     

Non-linear autoregressive time series with multivariate Gaussian mixtures as marginal distributions

A new form of non-linear autoregressive time series is proposed to model solar radiation data, by specifying joint marginal distributions at low lags to be multivariate Gaussian mixtures. The model is also a type of multiprocess dynamic linear model, but with the advantage that the likelihood has a closed form.     

BAYESIAN ANALYSIS OF VECTOR ARFIMA PROCESSES

A general framework is presented for Bayesian inference of multivariate time series exhibiting long-range dependence. The series are modelled using a vector autoregressive fractionally integrated moving-average (VARFIMA) process, which can capture both short-term correlation structure and long-range dependence characteristics of the individual series, as well as interdependence and feedback relationships between the series. To facilitate a sampling-based Bayesian approach, the exact joint posterior density is derived for the parameters, in a form that is computationally simpler than direct evaluation of the likelihood, and a modified Gibbs sampling algorithm is used to generate samples from the complete conditional distribution associated with each parameter. The paper also shows how an approximate form of the joint posterior density may be used for long time series. The procedure is illustrated using sea surface temperatures measured at three locations along the central California coast. These series are believed to be interdependent due to similarities in local atmospheric conditions at the different locations, and previous studies have found that they exhibit ‘long memory’ when studied individually. The approach adopted here permits investigation of the effects on model estimation of the interdependence and feedback relationships between the series.     

A semiparametric approach for modelling multivariate nonlinear time series

In this article, a semiparametric time‐varying nonlinear vector autoregressive (NVAR) model is proposed to model nonlinear vector time series data. We consider a combination of parametric and nonparametric estimation approaches to estimate the NVAR function for both independent and dependent errors. We use the multivariate Taylor series expansion of the link function up to the second order which has a parametric framework as a representation of the nonlinear vector regression function. After the unknown parameters are estimated by the maximum likelihood estimation procedure, the obtained NVAR function is adjusted by a nonparametric diagonal matrix, where the proposed adjusted matrix is estimated by the nonparametric kernel estimator. The asymptotic consistency properties of the proposed estimators are established. Simulation studies are conducted to evaluate the performance of the proposed semiparametric method. A real data example on short‐run interest rates and long‐run interest rates of United States Treasury securities is analyzed to demonstrate the application of the proposed approach. The Canadian Journal of Statistics 47: 668–687; 2019 © 2019 Statistical Society of Canada     

On hysteretic vector autoregressive model with applications

This paper proposes a new hysteretic vector autoregressive (HVAR) model in which the regime switching may be delayed when the hysteresis variable lies in a hysteresis zone. We integrate an adapted multivariate Student-t distribution from amending the scale mixtures of normal distributions. This HVAR model allows for a higher degree of flexibility in the degrees of freedom for each time series. We use the proposed model to test for a causal relationship between any two target time series. Using posterior odds ratios, we overcome the limitations of the classical approach to multiple testing. Both simulated and real examples herein help illustrate the suggested methods. We apply the proposed HVAR model to investigate the causal relationship between the quarterly growth rates of gross domestic product of United Kingdom and United States. Moreover, we check the pairwise lagged dependence of daily PM2.5 levels in three districts of Taipei.     

Screening for prostate cancer using multivariate mixed-effects models

Using several variables known to be related to prostate cancer, a multivariate classification method is developed to predict the onset of clinical prostate cancer. A multivariate mixed-effects model is used to describe longitudinal changes in prostate specific antigen (PSA), a free testosterone index (FTI), and body mass index (BMI) before any clinical evidence of prostate cancer. The patterns of change in these three variables are allowed to vary depending on whether the subject develops prostate cancer or not and the severity of the prostate cancer at diagnosis. An application of Bayes' theorem provides posterior probabilities that we use to predict whether an individual will develop prostate cancer and, if so, whether it is a high-risk or a low-risk cancer. The classification rule is applied sequentially one multivariate observation at a time until the subject is classified as a cancer case or until the last observation has been used. We perform the analyses using each of the three variables individually, combined together in pairs, and all three variables together in one analysis. We compare the classification results among the various analyses and a simulation study demonstrates how the sensitivity of prediction changes with respect to the number and type of variables used in the prediction process.     

Multivariate autoregressive time semes modeling: one scalar autoregressive model at-A-time

Multivariate (or interchangeably multichannel) autoregressive (MCAR) modeling of stationary and nonstationary time series data is achieved doing things one channel at-a-time using only scalar computations on instantaneous data. The one channel at-a-time modeling is achieved as an instantaneous response multichannel autoregressive model with orthogonal innovations variance. Conventional MCAR models are expressible as linear algebraic transformations of the instantaneous response orthogonal innovations models. By modeling multichannel time series one channel at-a-time, the problems of modeling multichannel time series are reduced to problems in the modeling of scalar autoregressive time series. The three longstanding time series modeling problems of achieving a relatively parsimonious MCAR representation, of multichannel stationary time series spectral estimation and of the modeling of nonstationary covariance time series are addressed using this paradigm.