Bayesian Identification of Multivariate Autoregressive Processes

Identification is one of the most important stages of a time series analysis. This paper develops a direct Bayesian technique to identify the order of multivariate autoregressive processes. By employing the conditional likelihood function and a matrix normal-Wishart prior density, or Jeffrey' vague prior, the proposed identification technique is based on deriving the exact posterior probability mass function of the model order in a convenient form. Then one may easily evaluate the posterior probabilities of the model order and choose the order that maximizes the posterior mass function to be the suitable order of the time series data being analyzed. Assuming the bivariate autoregressive processes, a numerical study, with different prior mass functions, is carried out to assess the efficiency of the proposed technique. The analysis of the numerical results supports the adequacy of the proposed technique in identifying the orders of multivariate autoregressive processes.     

Bayesian Identification of Seasonal Autoregressive Models

Abstract

A very important and essential phase of time series analysis is identifying the model orders. This article develops an approximate Bayesian procedure to identify the orders of seasonal autoregressive processes. Using either a normal-gamma prior density or a noninformative prior, which is combined with an approximate conditional likelihood function, the foundation of the proposed technique is to derive the joint posterior mass function of the model orders in an easy form. Then one may inspect the posterior mass function and choose the orders with the largest posterior probability to be the suitable orders of the time series being analyzed. A simulation study, with different priors mass functions, is carried out to test the adequacy of the proposed technique and compare it with some non-Bayesian automatic criteria. The analysis of the numerical results supports the adequacy of the proposed technique in identifying the orders of the autoregressive processes.     

Bayesian identification of double seasonal autoregressive time series models

ABSTRACT

Seasonal autoregressive (SAR) models have been modified and extended to model high frequency time series characterized by exhibiting double seasonal patterns. Some researchers have introduced Bayesian inference for double seasonal autoregressive (DSAR) models; however, none has tackled the problem of Bayesian identification of DSAR models. Therefore, in order to fill this gap, we present a Bayesian methodology to identify the order of DSAR models. Assuming the model errors are normally distributed and using three priors, i.e. natural conjugate, g, and Jeffreys’ priors, on the model parameters, we derive the joint posterior mass function of the model order in a closed-form. Accordingly, the posterior mass function can be investigated and the best order of DSAR model is chosen as a value with the highest posterior probability for the time series being analyzed. We evaluate the proposed Bayesian methodology using simulation study, and we then apply it to real-world hourly internet amount of traffic dataset.     

Bayesian classification with multivariate autoregressive sources that might have different orders

The main objective of this paper is to develop an exact Bayesian technique that can be used to assign a multivariate time series realization to one of several autoregressive sources, with unknown coefficients and precision, that might have different orders. The foundation of the proposed technique is to develop the posterior mass function of a classification vector, in an easy form, using the conditional likelihood function. A multivariate time series realization is assigned to the multivariate autoregressive source with the largest posterior probability. A simulation study, with uniform prior mass function, is carried out to demonstrate the performance of the proposed technique and to test its adequacy in handling the multivariate classification problems. The analysis of the numerical results supports the adequacy of the proposed technique in solving the classification problems with multivariate autoregressive sources.     

Bayesian inference for double SARMA models

In this paper, we present a Bayesian analysis of double seasonal autoregressive moving average models. We first consider the problem of estimating unknown lagged errors in the moving average part using non linear least squares method, and then using natural conjugate and Jeffreys’ priors we approximate the marginal posterior distributions to be multivariate t and gamma distributions for the model coefficients and precision, respectively. We evaluate the proposed Bayesian methodology using simulation study, and apply to real-world hourly electricity load data sets.     

Using the predictive distribution to determine control limits for the Bayesian MEWMA chart

Bayesian control charts have been proposed for monitoring multivariate processes with the multivariate exponentially weighted moving average (MEWMA) statistic. It has been suggested that we use limits based on the predictive distribution of the MEWMA statistic. This analysis, however is based on the erroneous result that the average run length (ARL) is a function of the exceedance probability, that is, the probability that the first point exceeds the control limit. We show how this result can be corrected and we discuss how the Bayesian MEWMA chart with limits based on the predictive distribution compares with other multivariate control chart procedures.     

Bayesian Inferences and Forecasts With Multiple Autoregressive Moving Average Models

The main objective of this paper is to develop convenient Bayesian techniques for estimation and forecasting which can be used to analyze multiple (multivariate) autoregressive moving average processes. Based on the conditional likelihood function and the least squares estimates of the residuals, the marginal posterior distribution of the coefficients of the model is approximated by a matrix t distribution, the marginal posterior distribution of the precision matrix is approximated by a Wishart distribution, and the predictive distribution is approximated by a multivariate t distribution. Some numerical examples are given to demonstrate the idea of using the proposed techniques to analyze different types of multiple ARMA models.     

On the identification of ARMA echelon-form models

An identification procedure for multivariate autoregressive moving average (ARMA) echelon-form models is proposed. It is based on the study of the linear dependence between rows of the Hankel matrix of serial correlations. To that end, we define a statistical test for checking the linear dependence between vectors of serial correlations. It is shown that the test statistic t?_n considered is distributed asymptotically as a finite linear combination of independent chi-square random variables with one degree of freedom under the null hypothesis, whereas under the alternative hypothesis, t?_N/N converges in probability to a positive constant. These results allow us, in particular, to compute the asymptotic probability of making a specification error with the proposed procedure. Links to other methods based on the application of canonical analysis are discussed. A simulation experiment was done in order to study the performance of the procedure. It is seen that the graphical representation of t?_N, as a function of N, can be very useful in identifying the dynamic structure of ARMA models. Furthermore, for the model considered, the proposed identification procedure performs very well for series of 100 observations or more and reasonably well with short series of 50 observations.     

Bayesian inferences and forecasts with moving averages processes

This paper will develop Bayesian inferential and forecasting techniques which can be used with any moving average process. By employing the conditional likelihood function, at-approximation to the predictive distribution and the marginal posterior distribution of the moving average parameters is developed. Several examples demonstrate posterior and predictive inferences.     

A Condition to Obtain the Same Decision in the Homogeneity Testing Problem from the Frequentist and Bayesian Point of View

We develop a Bayesian procedure for the homogeneity testing problem of r populations using r × s contingency tables. The posterior probability of the homogeneity null hypothesis is calculated using a mixed prior distribution. The methodology consists of choosing an appropriate value of π₀ for the mass assigned to the null and spreading the remainder, 1 ? π₀, over the alternative according to a density function. With this method, a theorem which shows when the same conclusion is reached from both frequentist and Bayesian points of view is obtained. A sufficient condition under which the p-value is less than a value α and the posterior probability is also less than 0.5 is provided.     

Bayesian inference and model comparison for asymmetric smooth transition heteroskedastic models

Inference, quantile forecasting and model comparison for an asymmetric double smooth transition heteroskedastic model is investigated. A Bayesian framework in employed and an adaptive Markov chain Monte Carlo scheme is designed. A mixture prior is proposed that alleviates the usual identifiability problem as the speed of transition parameter tends to zero, and an informative prior for this parameter is suggested, that allows for reliable inference and a proper posterior, despite the non-integrability of the likelihood function. A formal Bayesian posterior model comparison procedure is employed to compare the proposed model with its two limiting cases: the double threshold GARCH and symmetric ARX GARCH models. The proposed methods are illustrated using both simulated and international stock market return series. Some illustrations of the advantages of an adaptive sampling scheme for these models are also provided. Finally, Bayesian forecasting methods are employed in a Value-at-Risk study of the international return series. The results generally favour the proposed smooth transition model and highlight explosive and smooth nonlinear behaviour in financial markets.     

Bayesian estimation and the application of long memory stochastic volatility models

A new sampling-based Bayesian approach to the long memory stochastic volatility (LMSV) process is presented; the method is motivated by the GPH-estimator in fractionally integrated autoregressive moving average (ARFIMA) processes, which was originally proposed by J. Geweke and S. Porter-Hudak [The estimation and application of long memory time series models, Journal of Time Series Analysis, 4 (1983) 221–238]. In this work, we perform an estimation of the memory parameter in the Bayesian framework; an estimator is obtained by maximizing the posterior density of the memory parameter. Finally, we compare the GPH-estimator and the Bayes-estimator by means of a simulation study and our new approach is illustrated using several stock market indices; the new estimator is proved to be relatively stable for the various choices of frequencies used in the regression.     

MAP segmentation in Bayesian hidden Markov models: a case study

We consider the problem of estimating the maximum posterior probability (MAP) state sequence for a finite state and finite emission alphabet hidden Markov model (HMM) in the Bayesian setup, where both emission and transition matrices have Dirichlet priors. We study a training set consisting of thousands of protein alignment pairs. The training data is used to set the prior hyperparameters for Bayesian MAP segmentation. Since the Viterbi algorithm is not applicable any more, there is no simple procedure to find the MAP path, and several iterative algorithms are considered and compared. The main goal of the paper is to test the Bayesian setup against the frequentist one, where the parameters of HMM are estimated using the training data.     

Objective Bayesian multiple comparisons for normal variances

This paper considers the multiple comparisons problem for normal variances. We propose a solution based on a Bayesian model selection procedure to this problem in which no subjective input is considered. We construct the intrinsic and fractional priors for which the Bayes factors and model selection probabilities are well defined. The posterior probability of each model is used as a model selection tool. The behaviour of these Bayes factors is compared with the Bayesian information criterion of Schwarz and some frequentist tests.     

Order identification for gaussian moving averages using the covariation

In the traditional Box-Jenkins procedure for fitting ARMA time series models to data, the first step is order identification. The sample autocorrelation function can be used to identify pure moving average behavior. In this paper we consider using the autocovariation function identify the order of a univariate Gaussian time series. Simulation evidence indicates the suggested method may be a superior order identification tool when at least 100 observations are taken.     

Estimation of the intensity function of an inhomogeneous Poisson process with a change‐point

Recent work on point processes includes studying posterior convergence rates of estimating a continuous intensity function. In this article, convergence rates for estimating the intensity function and change‐point are derived for the more general case of a piecewise continuous intensity function. We study the problem of estimating the intensity function of an inhomogeneous Poisson process with a change‐point using non‐parametric Bayesian methods. An Markov Chain Monte Carlo (MCMC) algorithm is proposed to obtain estimates of the intensity function and the change‐point which is illustrated using simulation studies and applications. The Canadian Journal of Statistics 47: 604–618; 2019 © 2019 Statistical Society of Canada     

Testing equality of regression coefficients in heteroscedastic normal regression models

This article addresses the problem of testing whether the vectors of regression coefficients are equal for two independent normal regression models when the error variances are unknown. This problem poses severe difficulties both to the frequentist and Bayesian approaches to statistical inference. In the former approach, normal hypothesis testing theory does not apply because of the unrelated variances. In the latter, the prior distributions typically used for the parameters are improper and hence the Bayes factor-based solution cannot be used.We propose a Bayesian solution to this problem in which no subjective input is considered. We first generate “objective” proper prior distributions (intrinsic priors) for which the Bayes factor and model posterior probabilities are well defined. The posterior probability of each model is used as a model selection tool. This consistent procedure of testing hypotheses is compared with some of the frequentist approximate tests proposed in the literature.     

A local unit root test in mean for financial time series

A common financial trading strategy involves exploiting mean-reverting behaviour of paired asset prices. Since a unit root test can be used to determine which pairs of assets appear to exhibit mean-reverting behaviour, we propose a new Bayesian unit root to detect the presence of a local unit root vs. mean-reverting nonlinear smooth transition heteroskedastic alternative hypotheses. This test procedure is based on the posterior odds. For simultaneous estimation and inference, we employ an adaptive Bayesian Markov chain Monte Carlo scheme, which utilizes a mixture prior specification to solve the likelihood identification problem of the smoothing parameter and the autoregressive coefficient with a unit root. The size and power properties of the proposed method are examined via a simulation study. An empirical study examines the mean-reverting behaviour of price differential between stock and future.     

A data-driven selection of the number of clusters in the Dirichlet allocation model via Bayesian mixture modelling

In this paper, we consider a Bayesian mixture model that allows us to integrate out the weights of the mixture in order to obtain a procedure in which the number of clusters is an unknown quantity. To determine clusters and estimate parameters of interest, we develop an MCMC algorithm denominated by sequential data-driven allocation sampler. In this algorithm, a single observation has a non-null probability to create a new cluster and a set of observations may create a new cluster through the split-merge movements. The split-merge movements are developed using a sequential allocation procedure based in allocation probabilities that are calculated according to the Kullback–Leibler divergence between the posterior distribution using the observations previously allocated and the posterior distribution including a ‘new’ observation. We verified the performance of the proposed algorithm on the simulated data and then we illustrate its use on three publicly available real data sets.     

Bayesian hierarchical model for protein identifications

In proteomics, identification of proteins from complex mixtures of proteins extracted from biological samples is an important problem. Among the experimental technologies, mass spectrometry (MS) is the most popular one. Protein identification from MS data typically relies on a ‘two-step’ procedure of identifying the peptide first followed by the separate protein identification procedure next. In this setup, the interdependence of peptides and proteins is neglected resulting in relatively inaccurate protein identification. In this article, we propose a Markov chain Monte Carlo based Bayesian hierarchical model, a first of its kind in protein identification, which integrates the two steps and performs joint analysis of proteins and peptides using posterior probabilities. We remove the assumption of independence of proteins by using clustering group priors to the proteins based on the assumption that proteins sharing the same biological pathway are likely to be present or absent together and are correlated. The complete conditionals of the proposed joint model being tractable, we propose and implement a Gibbs sampling scheme for full posterior inference that provides the estimation and statistical uncertainties of all relevant parameters. The model has better operational characteristics compared to two existing ‘one-step’ procedures on a range of simulation settings as well as on two well-studied datasets.