Bayesian multiple structural change-points estimation in time series models with genetic algorithm

This article considers a time series model with a deterministic trend, in which multiple structural changes are explicitly taken into account, while the number and the location of change-points are unknown. We aim to figure out the best model with the appropriate number of change-points and a certain length of segments between points. We derive a posterior probability and then apply a genetic algorithm (GA) to calculate the posterior probabilities to locate the change-points. GA results in a powerful flexible tool which is shown to search over possible change-points. Numerical results obtained from simulation experiments show excellent empirical properties. To verify our model retrospectively, we estimate structural change-points with US and South Korean GDP data.     

Stochastic approximation Monte Carlo Gibbs sampling for structural change inference in a Bayesian heteroscedastic time series model

We consider a Bayesian deterministically trending dynamic time series model with heteroscedastic error variance, in which there exist multiple structural changes in level, trend and error variance, but the number of change-points and the timings are unknown. For a Bayesian analysis, a truncated Poisson prior and conjugate priors are used for the number of change-points and the distributional parameters, respectively. To identify the best model and estimate the model parameters simultaneously, we propose a new method by sequentially making use of the Gibbs sampler in conjunction with stochastic approximation Monte Carlo simulations, as an adaptive Monte Carlo algorithm. The numerical results are in favor of our method in terms of the quality of estimates.     

A Bayesian structural-change analysis via the stochastic approximation Monte Carlo and Gibbs sampler

In this article, we propose a Bayesian approach to estimate the multiple structural change-points in a level and the trend when the number of change-points is unknown. Our formulation of the structural-change model involves a binary discrete variable that indicates the structural change. The determination of the number and the form of structural changes are considered as a model selection issue in Bayesian structural-change analysis. We apply an advanced Monte Carlo algorithm, the stochastic approximation Monte Carlo (SAMC) algorithm, to this structural-change model selection issue. SAMC effectively functions for the complex structural-change model estimation, since it prevents entrapment in local posterior mode. The estimation of the model parameters in each regime is made using the Gibbs sampler after each change-point is detected. The performance of our proposed method has been investigated on simulated and real data sets, a long time series of US real gross domestic product, US uses of force between 1870 and 1994 and 1-year time series of temperature in Seoul, South Korea.     

A Bayesian multiple structural change regression model with autocorrelated errors

This paper develops a new Bayesian approach to change-point modeling that allows the number of change-points in the observed autocorrelated times series to be unknown. The model we develop assumes that the number of change-points have a truncated Poisson distribution. A genetic algorithm is used to estimate a change-point model, which allows for structural changes with autocorrelated errors. We focus considerable attention on the construction of autocorrelated structure for each regime and for the parameters that characterize each regime. Our techniques are found to work well in the simulation with a few change-points. An empirical analysis is provided involving the annual flow of the Nile River and the monthly total energy production in South Korea to lead good estimates for structural change-points.     

Detection of multiple undocumented change-points using adaptive Lasso

The problem of detecting multiple undocumented change-points in a historical temperature sequence with simple linear trend is formulated by a linear model. We apply adaptive least absolute shrinkage and selection operator (Lasso) to estimate the number and locations of change-points. Model selection criteria are used to choose the Lasso smoothing parameter. As adaptive Lasso may overestimate the number of change-points, we perform post-selection on change-points detected by adaptive Lasso using multivariate t simultaneous confidence intervals. Our method is demonstrated on the annual temperature data (year: 1902–2000) from Tuscaloosa, Alabama.     

A segmented regime-switching model with its application to stock market indices

This paper evaluates the ability of a Markov regime-switching log-normal (RSLN) model to capture the time-varying features of stock return and volatility. The model displays a better ability to depict a fat tail distribution as compared with using a log-normal model, which means that the RSLN model can describe observed market behavior better. Our major objective is to explore the capability of the model to capture stock market behavior over time. By analyzing the behavior of calibrated regime-switching parameters over different lengths of time intervals, the change-point concept is introduced and an algorithm is proposed for identifying the change-points in the series corresponding to the times when there are changes in parameter estimates. This algorithm for identifying change-points is tested on the Standard and Poor's 500 monthly index data from 1971 to 2008, and the Nikkei 225 monthly index data from 1984 to 2008. It is evident that the change-points we identify match the big events observed in the US stock market and the Japan stock market (e.g., the October 1987 stock market crash), and that the segmentations of stock index series, which are defined as the periods between change-points, match the observed bear–bull market phases.     

Exact Bayesian inference for off-line change-point detection in tree-structured graphical models

We consider the problem of change-point detection in multivariate time-series. The multivariate distribution of the observations is supposed to follow a graphical model, whose graph and parameters are affected by abrupt changes throughout time. We demonstrate that it is possible to perform exact Bayesian inference whenever one considers a simple class of undirected graphs called spanning trees as possible structures. We are then able to integrate on the graph and segmentation spaces at the same time by combining classical dynamic programming with algebraic results pertaining to spanning trees. In particular, we show that quantities such as posterior distributions for change-points or posterior edge probabilities over time can efficiently be obtained. We illustrate our results on both synthetic and experimental data arising from biology and neuroscience.     

Inference and prediction with arma processes

An essential ingredient of any time series analysis is the estimation of the model parameters and the forecasting of future observations. This investigation takes a Bayesian approach to the analysis of time series by making inferences of the model parameters from the posterior distribution and forecasting from the predictive distribution.

The foundation of the approach is to approximate the condi-tional likelihood by a normal-gamma distribution on the parameter space. The techniques illustrated with many examples of ARMA processes.     

Penalized least absolute deviations estimation for nonlinear model with change-points

This paper studies the asymptotic properties of a smoothed least absolute deviations estimator in a nonlinear parametric model with multiple change-points occurring at the unknown times with independent and identically distributed errors. The model is nonlinear in the sense that between two successive change-points the regression function is nonlinear into respect to parameters. It is shown via Monte Carlo simulations that its performance is competitive with that of least absolute deviations estimator and it is more efficient than the least squares estimator, particularly in the presence of the outlier points. If the number of change-points is unknown, an estimation criterion for this number is proposed. Interest of this method is that the objective function is approximated by a differentiable function and if the model contains outliers, it detects correctly the location of the change-points.     

Sampling the Future: A Bayesian Approach to Forecasting From Univariate Time Series Models

The Box–Jenkins methodology for modeling and forecasting from univariate time series models has long been considered a standard to which other forecasting techniques have been compared. To a Bayesian statistician, however, the method lacks an important facet—a provision for modeling uncertainty about parameter estimates. We present a technique called sampling the future for including this feature in both the estimation and forecasting stages. Although it is relatively easy to use Bayesian methods to estimate the parameters in an autoregressive integrated moving average (ARIMA) model, there are severe difficulties in producing forecasts from such a model. The multiperiod predictive density does not have a convenient closed form, so approximations are needed. In this article, exact Bayesian forecasting is approximated by simulating the joint predictive distribution. First, parameter sets are randomly generated from the joint posterior distribution. These are then used to simulate future paths of the time series. This bundle of many possible realizations is used to project the future in several ways. Highest probability forecast regions are formed and portrayed with computer graphics. The predictive density's shape is explored. Finally, we discuss a method that allows the analyst to subjectively modify the posterior distribution on the parameters and produce alternate forecasts.     

Importance sampling for signal extraction model in time series analysis

In time series analysis, signal extraction model (SEM) is used to estimate unobserved signal component from observed time series data. Since parameters of the components in SEM are often unknown in practice, a commonly used method is to estimate unobserved signal component using the maximum likelihood estimates (MLEs) of parameters of the components. This paper explores an alternative way to estimate unobserved signal component when parameters of the components are unknown. The suggested method makes use of importance sampling (IS) with Bayesian inference. The basic idea is to treat parameters of the components in SEM as a random vector and compute a posterior probability density function of the parameters using Bayesian inference. Then IS method is applied to integrate out the parameters and thus estimates of unobserved signal component, unconditional to the parameters, can be obtained. This method is illustrated with a real time series data. Then a Monte Carlo study with four different types of time series models is carried out to compare a performance of this method with that of a commonly used method. The study shows that IS method with Bayesian inference is computationally feasible and robust, and more efficient in terms of mean square errors (MSEs) than a commonly used method.     

Modelling population-based cancer survival trends using join point models for grouped survival data

Summary.  In the USA cancer as a whole is the second leading cause of death and a major burden to health care; thus medical progress against cancer is a major public health goal. There are many individual studies to suggest that cancer treatment breakthroughs and early diagnosis have significantly improved the prognosis of cancer patients. To understand better the relationship between medical improvements and the survival experience for the patient population at large, it is useful to evaluate cancer survival trends on the population level, e.g. to find out when and how much the cancer survival rates changed. We analyse population-based grouped cancer survival data by incorporating join points into the survival models. A join point survival model facilitates the identification of trends with significant change-points in cancer survival, when related to cancer treatments or interventions. The Bayesian information criterion is used to select the number of join points. The performance of the join point survival models is evaluated with respect to cancer prognosis, join point locations, annual percentage changes in death rates by year of diagnosis and sample sizes through intensive simulation studies. The model is then applied to grouped relative survival data for several major cancer sites from the 'Surveillance, epidemiology and end results' programme of the National Cancer Institute. The change-points in the survival trends for several major cancer sites are identified and the potential driving forces behind such change-points are discussed.     

Autoregressive Moving Average Infinite Hidden Markov-Switching Models

Markov-switching models are usually specified under the assumption that all the parameters change when a regime switch occurs. Relaxing this hypothesis and being able to detect which parameters evolve over time is relevant for interpreting the changes in the dynamics of the series, for specifying models parsimoniously, and may be helpful in forecasting. We propose the class of sticky infinite hidden Markov-switching autoregressive moving average models, in which we disentangle the break dynamics of the mean and the variance parameters. In this class, the number of regimes is possibly infinite and is determined when estimating the model, thus avoiding the need to set this number by a model choice criterion. We develop a new Markov chain Monte Carlo estimation method that solves the path dependence issue due to the moving average component. Empirical results on macroeconomic series illustrate that the proposed class of models dominates the model with fixed parameters in terms of point and density forecasts.     

Exact posterior distributions and model selection criteria for multiple change-point detection problems

In segmentation problems, inference on change-point position and model selection are two difficult issues due to the discrete nature of change-points. In a Bayesian context, we derive exact, explicit and tractable formulae for the posterior distribution of variables such as the number of change-points or their positions. We also demonstrate that several classical Bayesian model selection criteria can be computed exactly. All these results are based on an efficient strategy to explore the whole segmentation space, which is very large. We illustrate our methodology on both simulated data and a comparative genomic hybridization profile.     

A Bayesian model for multiple change point to extremes,with application to environmental and financial data

Abrupt changes often occur for environmental and financial time series. Most often, these changes are due to human intervention. Change point analysis is a statistical tool used to analyze sudden changes in observations along the time series. In this paper, we propose a Bayesian model for extreme values for environmental and economic datasets that present a typical change point behavior. The model proposed in this paper addresses the situation in which more than one change point can occur in a time series. By analyzing maxima, the distribution of each regime is a generalized extreme value distribution. In this model, the change points are unknown and considered parameters to be estimated. Simulations of extremes with two change points showed that the proposed algorithm can recover the true values of the parameters, in addition to detecting the true change points in different configurations. Also, the number of change points was a problem to be considered, and the Bayesian estimation can correctly identify the correct number of change points for each application. Environmental and financial data were analyzed and results showed the importance of considering the change point in the data and revealed that this change of regime brought about an increase in the return levels, increasing the number of floods in cities around the rivers. Stock market levels showed the necessity of a model with three different regimes.     

Bayesian variable selection in Poisson change-point regression analysis

In this article, we develop a Bayesian variable selection method that concerns selection of covariates in the Poisson change-point regression model with both discrete and continuous candidate covariates. Ranging from a null model with no selected covariates to a full model including all covariates, the Bayesian variable selection method searches the entire model space, estimates posterior inclusion probabilities of covariates, and obtains model averaged estimates on coefficients to covariates, while simultaneously estimating a time-varying baseline rate due to change-points. For posterior computation, the Metropolis-Hastings within partially collapsed Gibbs sampler is developed to efficiently fit the Poisson change-point regression model with variable selection. We illustrate the proposed method using simulated and real datasets.     

Priors and component structures in autoregressive time series models

New approaches to prior specification and structuring in autoregressive time series models are introduced and developed. We focus on defining classes of prior distributions for parameters and latent variables related to latent components of an autoregressive model for an observed time series. These new priors naturally permit the incorporation of both qualitative and quantitative prior information about the number and relative importance of physically meaningful components that represent low frequency trends, quasi-periodic subprocesses and high frequency residual noise components of observed series. The class of priors also naturally incorporates uncertainty about model order and hence leads in posterior analysis to model order assessment and resulting posterior and predictive inferences that incorporate full uncertainties about model order as well as model parameters. Analysis also formally incorporates uncertainty and leads to inferences about unknown initial values of the time series, as it does for predictions of future values. Posterior analysis involves easily implemented iterative simulation methods, developed and described here. One motivating field of application is climatology, where the evaluation of latent structure, especially quasi-periodic structure, is of critical importance in connection with issues of global climatic variability. We explore the analysis of data from the southern oscillation index, one of several series that has been central in recent high profile debates in the atmospheric sciences about recent apparent trends in climatic indicators.     

A Cox-type regression model with change-points in the covariates

We consider a Cox-type regression model with change-points in the covariates. A change-point specifies the unknown threshold at which the influence of a covariate shifts smoothly, i.e., the regression parameter may change over the range of a covariate and the underlying regression function is continuous but not differentiable. The model can be used to describe change-points in different covariates but also to model more than one change-point in a single covariate. Estimates of the change-points and of the regression parameters are derived and their properties are investigated. It is shown that not only the estimates of the regression parameters are [Formula: see text] -consistent but also the estimates of the change-points in contrast to the conjecture of other authors. Asymptotic normality is shown by using results developed for M-estimators. At the end of this paper we apply our model to an actuarial dataset, the PBC dataset of Fleming and Harrington (Counting processes and survival analysis, 1991) and to a dataset of electric motors.     

Bayesian experimental design for a toxicokinetic-toxicodynamic model

The aim of this study is to apply the Bayesian method of identifying optimal experimental designs to a toxicokinetic-toxicodynamic model that describes the response of aquatic organisms to time dependent concentrations of toxicants. As for experimental designs, we restrict ourselves to pulses and constant concentrations. A design of an experiment is called optimal within this set of designs if it maximizes the expected gain of knowledge about the parameters. Focus is on parameters that are associated with the auxiliary damage variable of the model that can only be inferred indirectly from survival time series data. Gain of knowledge through an experiment is quantified both with the ratio of posterior to prior variances of individual parameters and with the entropy of the posterior distribution relative to the prior on the whole parameter space. The numerical methods developed to calculate expected gain of knowledge are expected to be useful beyond this case study, in particular for multinomially distributed data such as survival time series data.     

BAYESIAN ANALYSIS OF VECTOR ARFIMA PROCESSES

A general framework is presented for Bayesian inference of multivariate time series exhibiting long-range dependence. The series are modelled using a vector autoregressive fractionally integrated moving-average (VARFIMA) process, which can capture both short-term correlation structure and long-range dependence characteristics of the individual series, as well as interdependence and feedback relationships between the series. To facilitate a sampling-based Bayesian approach, the exact joint posterior density is derived for the parameters, in a form that is computationally simpler than direct evaluation of the likelihood, and a modified Gibbs sampling algorithm is used to generate samples from the complete conditional distribution associated with each parameter. The paper also shows how an approximate form of the joint posterior density may be used for long time series. The procedure is illustrated using sea surface temperatures measured at three locations along the central California coast. These series are believed to be interdependent due to similarities in local atmospheric conditions at the different locations, and previous studies have found that they exhibit ‘long memory’ when studied individually. The approach adopted here permits investigation of the effects on model estimation of the interdependence and feedback relationships between the series.