A Bayesian structural-change analysis via the stochastic approximation Monte Carlo and Gibbs sampler

In this article, we propose a Bayesian approach to estimate the multiple structural change-points in a level and the trend when the number of change-points is unknown. Our formulation of the structural-change model involves a binary discrete variable that indicates the structural change. The determination of the number and the form of structural changes are considered as a model selection issue in Bayesian structural-change analysis. We apply an advanced Monte Carlo algorithm, the stochastic approximation Monte Carlo (SAMC) algorithm, to this structural-change model selection issue. SAMC effectively functions for the complex structural-change model estimation, since it prevents entrapment in local posterior mode. The estimation of the model parameters in each regime is made using the Gibbs sampler after each change-point is detected. The performance of our proposed method has been investigated on simulated and real data sets, a long time series of US real gross domestic product, US uses of force between 1870 and 1994 and 1-year time series of temperature in Seoul, South Korea.     

Piecewise FARIMA models for long-memory time series

We consider the problem of modelling a long-memory time series using piecewise fractional autoregressive integrated moving average processes. The number as well as the locations of structural break points (BPs) and the parameters of each regime are assumed to be unknown. A four-step procedure is proposed to find out the BPs and to estimate the parameters of each regime. Its effectiveness is shown by Monte Carlo simulations and an application to real traffic data modelling is considered.     

Bayesian model selection and parameter estimation for possibly asymmetric and non-stationary time series using a reversible jump Markov chain Monte Carlo approach

A Markov chain Monte Carlo (MCMC) approach, called a reversible jump MCMC, is employed in model selection and parameter estimation for possibly non-stationary and non-linear time series data. The non-linear structure is modelled by the asymmetric momentum threshold autoregressive process (MTAR) of Enders & Granger (1998) or by the asymmetric self-exciting threshold autoregressive process (SETAR) of Tong (1990). The non-stationary and non-linear feature is represented by the MTAR (or SETAR) model in which one ( 𝜌 1 ) of the AR coefficients is greater than one, and the other ( 𝜌 2 ) is smaller than one. The other non-stationary and linear, stationary and nonlinear, and stationary and linear features, represented respectively by ( 𝜌 1 = 𝜌 2 = 1 ), ( 𝜌 1 p 𝜌 2 < 1 ) and ( 𝜌 1 = 𝜌 2 < 1 ), are also considered as possible models. The reversible jump MCMC provides estimates of posterior probabilities for these four different models as well as estimates of the AR coefficients 𝜌 1 and 𝜌 2 . The proposed method is illustrated by analysing six series of US interest rates in terms of model selection, parameter estimation, and forecasting.     

Lasso penalized semiparametric regression on high-dimensional recurrent event data via coordinate descent

This paper studies a fast computational algorithm for variable selection on high-dimensional recurrent event data. Based on the lasso penalized partial likelihood function for the response process of recurrent event data, a coordinate descent algorithm is used to accelerate the estimation of regression coefficients. This algorithm is capable of selecting important predictors for underdetermined problems where the number of predictors far exceeds the number of cases. The selection strength is controlled by a tuning constant that is determined by a generalized cross-validation method. Our numerical experiments on simulated and real data demonstrate the good performance of penalized regression in model building for recurrent event data in high-dimensional settings.     

A generalised NGINAR(1) process with inflated‐parameter geometric counting series

In this paper we propose a new stationary first‐order non‐negative integer valued autoregressive process with geometric marginals based on a generalised version of the negative binomial thinning operator. In this manner we obtain another process that we refer to as a generalised stationary integer‐valued autoregressive process of the first order with geometric marginals. This new process will enable one to tackle the problem of overdispersion inherent in the analysis of integer‐valued time series data, and contains the new geometric process as a particular case. In addition various properties of the new process, such as conditional distribution, autocorrelation structure and innovation structure, are derived. We discuss conditional maximum likelihood estimation of the model parameters. We evaluate the performance of the conditional maximum likelihood estimators by a Monte Carlo study. The proposed process is fitted to time series of number of weekly sales (economics) and weekly number of syphilis cases (medicine) illustrating its capabilities in challenging cases of highly overdispersed count data.     

Classical model selection via simulated annealing

Summary. The classical approach to statistical analysis is usually based upon finding values for model parameters that maximize the likelihood function. Model choice in this context is often also based on the likelihood function, but with the addition of a penalty term for the number of parameters. Though models may be compared pairwise by using likelihood ratio tests for example, various criteria such as the Akaike information criterion have been proposed as alternatives when multiple models need to be compared. In practical terms, the classical approach to model selection usually involves maximizing the likelihood function associated with each competing model and then calculating the corresponding criteria value(s). However, when large numbers of models are possible, this quickly becomes infeasible unless a method that simultaneously maximizes over both parameter and model space is available. We propose an extension to the traditional simulated annealing algorithm that allows for moves that not only change parameter values but also move between competing models. This transdimensional simulated annealing algorithm can therefore be used to locate models and parameters that minimize criteria such as the Akaike information criterion, but within a single algorithm, removing the need for large numbers of simulations to be run. We discuss the implementation of the transdimensional simulated annealing algorithm and use simulation studies to examine its performance in realistically complex modelling situations. We illustrate our ideas with a pedagogic example based on the analysis of an autoregressive time series and two more detailed examples: one on variable selection for logistic regression and the other on model selection for the analysis of integrated recapture–recovery data.     

On a mixture vector autoregressive model

The authors show how to extend univariate mixture autoregressive models to a multivariate time series context. Similar to the univariate case, the multivariate model consists of a mixture of stationary or nonstationary autoregressive components. The authors give the first and second order stationarity conditions for a multivariate case up to order 2. They also derive the second order stationarity condition for the univariate mixture model up to arbitrary order. They describe an EM algorithm for estimation, as well as a diagnostic checking procedure. They study the performance of their method via simulations and include a real application.     

A significance test for classifying arma models

Given that the Euclidean distance between the parameter estimates of autoregressive expansions of autoregressive moving average models can be used to classify stationary time series into groups, a test of hypothesis is proposed to determine whether two stationary series in a particular group have significantly different generating processes. Based on this test a new clustering algorithm is also proposed. The results of Monte Carlo simulations are given.     

Periodic integration: further results on model selection and forecasting

This paper considers model selection and forecasting issues in two closely related models for nonstationary periodic autoregressive time series [PAR]. Periodically integrated seasonal time series [PIAR] need a periodic differencing filter to remove the stochastic trend. On the other hand, when the nonperiodic first order differencing filter can be applied, one can have a periodic model with a nonseasonal unit root [PARI]. In this paper, we discuss and evaluate two testing strategies to select between these two models. Furthermore, we compare the relative forecasting performance of each model using Monte Carlo simulations and some U.K. macroeconomic seasonal time series. One result is that forecasting with PARI models while the data generating process is a PIAR process seems to be worse thanvice versa.     

A novel bagging approach for variable ranking and selection via a mixed importance measure

At present, ensemble learning has exhibited its great power in stabilizing and enhancing the performance of some traditional variable selection methods such as lasso and genetic algorithm. In this paper, a novel bagging ensemble method called BSSW is developed to implement variable ranking and selection in linear regression models. Its main idea is to execute stepwise search algorithm on multiple bootstrap samples. In each trial, a mixed importance measure is assigned to each variable according to the order that it is selected into final model as well as the improvement of model fitting resulted from its inclusion. Based on the importance measure averaged across some bootstrapping trials, all candidate variables are ranked and then decided to be important or not. To extend the scope of application, BSSW is extended to the situation of generalized linear models. Experiments carried out with some simulated and real data indicate that BSSW achieves better performance in most studied cases when compared with several other existing methods.     

On a mixture autoregressive model

We generalize the Gaussian mixture transition distribution (GMTD) model introduced by Le and co-workers to the mixture autoregressive (MAR) model for the modelling of non-linear time series. The models consist of a mixture of K stationary or non-stationary AR components. The advantages of the MAR model over the GMTD model include a more full range of shape changing predictive distributions and the ability to handle cycles and conditional heteroscedasticity in the time series. The stationarity conditions and autocorrelation function are derived. The estimation is easily done via a simple EM algorithm and the model selection problem is addressed. The shape changing feature of the conditional distributions makes these models capable of modelling time series with multimodal conditional distributions and with heteroscedasticity. The models are applied to two real data sets and compared with other competing models. The MAR models appear to capture features of the data better than other competing models do.     

Multivariate autoregressive time semes modeling: one scalar autoregressive model at-A-time

Multivariate (or interchangeably multichannel) autoregressive (MCAR) modeling of stationary and nonstationary time series data is achieved doing things one channel at-a-time using only scalar computations on instantaneous data. The one channel at-a-time modeling is achieved as an instantaneous response multichannel autoregressive model with orthogonal innovations variance. Conventional MCAR models are expressible as linear algebraic transformations of the instantaneous response orthogonal innovations models. By modeling multichannel time series one channel at-a-time, the problems of modeling multichannel time series are reduced to problems in the modeling of scalar autoregressive time series. The three longstanding time series modeling problems of achieving a relatively parsimonious MCAR representation, of multichannel stationary time series spectral estimation and of the modeling of nonstationary covariance time series are addressed using this paradigm.     

Statistical Analysis Of Mixture Vector Autoregressive Models

In this paper, we reconsider the mixture vector autoregressive model, which was proposed in the literature for modelling non‐linear time series. We complete and extend the stationarity conditions, derive a matrix formula in closed form for the autocovariance function of the process and prove a result on stable vector autoregressive moving‐average representations of mixture vector autoregressive models. For these results, we apply techniques related to a Markovian representation of vector autoregressive moving‐average processes. Furthermore, we analyse maximum likelihood estimation of model parameters by using the expectation–maximization algorithm and propose a new iterative algorithm for getting the maximum likelihood estimates. Finally, we study the model selection problem and testing procedures. Several examples, simulation experiments and an empirical application based on monthly financial returns illustrate the proposed procedures.     

Bent‐cable regression with autoregressive noise

Motivated by time series of atmospheric concentrations of certain pollutants the authors develop bent‐cable regression for autocorrelated errors. Bent‐cable regression extends the popular piecewise linear (broken‐stick) model, allowing for a smooth change region of any non‐negative width. Here the authors consider autoregressive noise added to a bent‐cable mean structure, with unknown regression and time series parameters. They develop asymptotic theory for conditional least‐squares estimation in a triangular array framework, wherein each segment of the bent cable contains an increasing number of observations while the autoregressive order remains constant as the sample size grows. They explore the theory in a simulation study, develop implementation details, apply the methodology to the motivating pollutant dataset, and provide a scientific interpretation of the bent‐cable change point not discussed previously. The Canadian Journal of Statistics 38: 386–407; 2010 © 2010 Statistical Society of Canada     

Autoregressiv model identification for multivariate time series

A procedure is developed for the identification of autoregressive models for stationary invertible multivariate Gaussian time series. Model selection is based on either the AIC information criterion or on a statistic called CVR, cross-validatory residual sum of squares. An example is given to show that the forecasts generated by these models compare favorably with those generated by other common time series modeling techniques.     

A new model selection procedure for finite mixture regression models

Abstract

In this article, we propose a new penalized-likelihood method to conduct model selection for finite mixture of regression models. The penalties are imposed on mixing proportions and regression coefficients, and hence order selection of the mixture and the variable selection in each component can be simultaneously conducted. The consistency of order selection and the consistency of variable selection are investigated. A modified EM algorithm is proposed to maximize the penalized log-likelihood function. Numerical simulations are conducted to demonstrate the finite sample performance of the estimation procedure. The proposed methodology is further illustrated via real data analysis.     

Scalable spatio‐temporal Bayesian analysis of high‐dimensional electroencephalography data

We present a scalable Bayesian modelling approach for identifying brain regions that respond to a certain stimulus and use them to classify subjects. More specifically, we deal with multi‐subject electroencephalography (EEG) data with a binary response distinguishing between alcoholic and control groups. The covariates are matrix‐variate with measurements taken from each subject at different locations across multiple time points. EEG data have a complex structure with both spatial and temporal attributes. We use a divide‐and‐conquer strategy and build separate local models, that is, one model at each time point. We employ Bayesian variable selection approaches using a structured continuous spike‐and‐slab prior to identify the locations that respond to a certain stimulus. We incorporate the spatio‐temporal structure through a Kronecker product of the spatial and temporal correlation matrices. We develop a highly scalable estimation algorithm, using likelihood approximation, to deal with large number of parameters in the model. Variable selection is done via clustering of the locations based on their duration of activation. We use scoring rules to evaluate the prediction performance. Simulation studies demonstrate the efficiency of our scalable algorithm in terms of estimation and fast computation. We present results using our scalable approach on a case study of multi‐subject EEG data.     

Oracle GMM estimation for misspecified models via thresholding

We propose a thresholding generalized method of moments (GMM) estimator for misspecified time series moment condition models. This estimator has the following oracle property: its asymptotic behavior is the same as of any efficient GMM estimator obtained under the a priori information that the true model were known. We propose data adaptive selection methods for thresholding parameter using multiple testing procedures. We determine the limiting null distributions of classical parameter tests and show the consistency of the corresponding block-bootstrap tests used in conjunction with thresholding GMM inference. We present the results of a simulation study for a misspecified instrumental variable regression model and for a vector autoregressive model with measurement error. We illustrate an application of the proposed methodology to data analysis of a real-world dataset.     

Testing for coefficient stability of AR(1) model when the null is an integrated or a stationary process

In this paper, we propose a new test for coefficient stability of an AR(1) model against the random coefficient autoregressive model of order 1 neither assuming a stationary nor a non-stationary process under the null hypothesis of a constant coefficient. The proposed test is obtained as a modification of the locally best invariant (LBI) test by Lee [(1998). Coefficient constancy test in a random coefficient autoregressive model. J. Statist. Plann. Inference 74, 93–101]. We examine finite sample properties of the proposed test by Monte Carlo experiments comparing with other existing tests, in particular, the LBI test by McCabe and Tremayne [(1995). Testing a time series for difference stationary. Ann. Statist. 23 (3), 1015–1028], which is for the null of a unit root process against the alternative of a stochastic unit root process.