Reversible jump, birth-and-death and more general continuous time Markov chain Monte Carlo samplers

Summary. Reversible jump methods are the most commonly used Markov chain Monte Carlo tool for exploring variable dimension statistical models. Recently, however, an alternative approach based on birth-and-death processes has been proposed by Stephens for mixtures of distributions. We show that the birth-and-death setting can be generalized to include other types of continuous time jumps like split-and-combine moves in the spirit of Richardson and Green. We illustrate these extensions both for mixtures of distributions and for hidden Markov models. We demonstrate the strong similarity of reversible jump and continuous time methodologies by showing that, on appropriate rescaling of time, the reversible jump chain converges to a limiting continuous time birth-and-death process. A numerical comparison in the setting of mixtures of distributions highlights this similarity.     

Rate estimation in partially observed Markov jump processes with measurement errors

We present a simulation methodology for Bayesian estimation of rate parameters in Markov jump processes arising for example in stochastic kinetic models. To handle the problem of missing components and measurement errors in observed data, we embed the Markov jump process into the framework of a general state space model. We do not use diffusion approximations. Markov chain Monte Carlo and particle filter type algorithms are introduced which allow sampling from the posterior distribution of the rate parameters and the Markov jump process also in data-poor scenarios. The algorithms are illustrated by applying them to rate estimation in a model for prokaryotic auto-regulation and the stochastic Oregonator, respectively.     

Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions

Summary. The major implementational problem for reversible jump Markov chain Monte Carlo methods is that there is commonly no natural way to choose jump proposals since there is no Euclidean structure in the parameter space to guide our choice. We consider mechanisms for guiding the choice of proposal. The first group of methods is based on an analysis of acceptance probabilities for jumps. Essentially, these methods involve a Taylor series expansion of the acceptance probability around certain canonical jumps and turn out to have close connections to Langevin algorithms. The second group of methods generalizes the reversible jump algorithm by using the so-called saturated space approach. These allow the chain to retain some degree of memory so that, when proposing to move from a smaller to a larger model, information is borrowed from the last time that the reverse move was performed. The main motivation for this paper is that, in complex problems, the probability that the Markov chain moves between such spaces may be prohibitively small, as the probability mass can be very thinly spread across the space. Therefore, finding reasonable jump proposals becomes extremely important. We illustrate the procedure by using several examples of reversible jump Markov chain Monte Carlo applications including the analysis of autoregressive time series, graphical Gaussian modelling and mixture modelling.     

A Spectral Estimator of Arma Parameters from Thresholded Data

We consider computationally-fast methods for estimating parameters in ARMA processes from binary time series data, obtained by thresholding the latent ARMA process. All methods involve matching estimated and expected autocorrelations of the binary series. In particular, we focus on the spectral representation of the likelihood of an ARMA process and derive a restricted form of this likelihood, which uses correlations at only the first few lags. We contrast these methods with an efficient but computationally-intensive Markov chain Monte Carlo (MCMC) method. In a simulation study we show that, for a range of ARMA processes, the spectral method is more efficient than variants of least squares and much faster than MCMC. We illustrate by fitting an ARMA(2,1) model to a binary time series of cow feeding data.     

ON LIKELIHOOD ESTIMATION FOR DISCRETELY OBSERVED MARKOV JUMP PROCESSES

The parameter estimation problem for a Markov jump process sampled at equidistant time points is considered here. Unlike the diffusion case where a closed form of the likelihood function is usually unavailable, here an explicit expansion of the likelihood function of the sampled chain is provided. Under suitable ergodicity conditions on the jump process, the consistency and the asymptotic normality of the likelihood estimator are established as the observation period tends to infinity. Simulation experiments are conducted to demonstrate the computational facility of the method.     

Integer autoregressive models with structural breaks

Even though integer-valued time series are common in practice, the methods for their analysis have been developed only in recent past. Several models for stationary processes with discrete marginal distributions have been proposed in the literature. Such processes assume the parameters of the model to remain constant throughout the time period. However, this need not be true in practice. In this paper, we introduce non-stationary integer-valued autoregressive (INAR) models with structural breaks to model a situation, where the parameters of the INAR process do not remain constant over time. Such models are useful while modelling count data time series with structural breaks. The Bayesian and Markov Chain Monte Carlo (MCMC) procedures for the estimation of the parameters and break points of such models are discussed. We illustrate the model and estimation procedure with the help of a simulation study. The proposed model is applied to the two real biometrical data sets.     

Matrix-analytic Models and their Analysis

We survey phase-type distributions and Markovian point processes, aspects of how to use such models in applied probability calculations and how to fit them to observed data. A phase-type distribution is defined as the time to absorption in a finite continuous time Markov process with one absorbing state. This class of distributions is dense and contains many standard examples like all combinations of exponential in series/parallel. A Markovian point process is governed by a finite continuous time Markov process (typically ergodic), such that points are generated at a Poisson intensity depending on the underlying state and at transitions; a main special case is a Markov-modulated Poisson process. In both cases, the analytic formulas typically contain matrix-exponentials, and the matrix formalism carried over when the models are used in applied probability calculations as in problems in renewal theory, random walks and queueing. The statistical analysis is typically based upon the EM algorithm, viewing the whole sample path of the background Markov process as the latent variable.     

A Monte Carlo method for filtering a marked doubly stochastic Poisson process

The author provides an approximated solution for the filtering of a state-space model, where the hidden state process is a continuous-time pure jump Markov process and the observations come from marked point processes. Each state k corresponds to a different marked point process, defined by its conditional intensity function λ _k (t). When a state is visited by the hidden process, the corresponding marked point process is observed. The filtering equations are obtained by applying the innovation method and the integral representation theorem of a point process martingale. Since the filtering equations belong to the family of Kushner–Stratonovich equations, an iterative solution is calculated. The theoretical solution is approximated and a Monte Carlo integration technique employed to implement it. The sequential method has been tested on a simulated data set based on marked point processes widely used in the statistical analysis of seismic sequences: the Poisson model, the stress release model and the Etas model.     

Sequential Bayesian inference in hidden Markov stochastic kinetic models with application to detection and response to seasonal epidemics

We study sequential Bayesian inference in stochastic kinetic models with latent factors. Assuming continuous observation of all the reactions, our focus is on joint inference of the unknown reaction rates and the dynamic latent states, modeled as a hidden Markov factor. Using insights from nonlinear filtering of continuous-time jump Markov processes we develop a novel sequential Monte Carlo algorithm for this purpose. Our approach applies the ideas of particle learning to minimize particle degeneracy and exploit the analytical jump Markov structure. A motivating application of our methods is modeling of seasonal infectious disease outbreaks represented through a compartmental epidemic model. We demonstrate inference in such models with several numerical illustrations and also discuss predictive analysis of epidemic countermeasures using sequential Bayes estimates.     

Variable dimension via stochastic volatility model using FX rates

In this paper, changepoint analysis is applied to stochastic volatility (SV) models which aim to understand the locations and movements of high frequency FX financial time series. Bayesian inference using the Markov Chain Monte Carlo method is performed using a process called variable dimension for SV parameters. Interesting results are that FX series have locations where one or more positions of the sequence correspond to systemic changes, and overall non-stationarity, in the returns process. Furthermore, we found that the changepoint locations provide an informative estimate for all FX series. Importantly in most cases, the detected changepoints can be identified with economic factors relevant to the country concerned. This helps support the fact that macroeconomics news and the movement in financial price are positively related.     

Non‐Parametric Estimation of the Conditional Distribution of the Interjumping Times for Piecewise‐Deterministic Markov Processes

This paper presents a non‐parametric method for estimating the conditional density associated to the jump rate of a piecewise‐deterministic Markov process. In our framework, the estimation needs only one observation of the process within a long time interval. Our method relies on a generalization of Aalen's multiplicative intensity model. We prove the uniform consistency of our estimator, under some reasonable assumptions related to the primitive characteristics of the process. A simulation study illustrates the behaviour of our estimator.     

Quasi- and pseudo-maximum likelihood estimators for discretely observed continuous-time Markov branching processes

This article deals with quasi- and pseudo-likelihood estimation for a class of continuous-time multi-type Markov branching processes observed at discrete points in time. “Conventional” and conditional estimation are discussed for both approaches. We compare their properties and identify situations where they lead to asymptotically equivalent estimators. Both approaches possess robustness properties, and coincide with maximum likelihood estimation in some cases. Quasi-likelihood functions involving only linear combinations of the data may be unable to estimate all model parameters. Remedial measures exist, including the resort either to non-linear functions of the data or to conditioning the moments on appropriate sigma-algebras. The method of pseudo-likelihood may also resolve this issue. We investigate the properties of these approaches in three examples: the pure birth process, the linear birth-and-death process, and a two-type process that generalizes the previous two examples. Simulations studies are conducted to evaluate performance in finite samples.     

Characterization of discrete scale invariant Markov sequences

ABSTRACT

Some special sampling of discrete scale invariant (DSI) processes are presented to provide a multi-dimensional self-similar process in correspondence. By imposing Markov property we show that the covariance functions of such Markov DSI sequences are characterized by variance, and covariance of adjacent samples in the first scale interval. We also provide a theoretical method for estimating spectral density matrix of corresponding multi-dimensional self-similar Markov process. Some examples such as simple Brownian motion (sBm) with drift and scale invariant autoregressive model are presented and these properties are investigated. We present two new method to estimate Hurst parameter of DSI processes and apply them to some sBm and also to the SP500 indices for some period which has DSI property. We compare our estimates with the maximum-likelihood and rescaled range (R/S) method which are applied to the corresponding multi-dimensional self-similar processes.     

A New Approach to Volatility Modeling: The Factorial Hidden Markov Volatility Model

A new process—the factorial hidden Markov volatility (FHMV) model—is proposed to model financial returns or realized variances. Its dynamics are driven by a latent volatility process specified as a product of three components: a Markov chain controlling volatility persistence, an independent discrete process capable of generating jumps in the volatility, and a predictable (data-driven) process capturing the leverage effect. An economic interpretation is attached to each one of these components. Moreover, the Markov chain and jump components allow volatility to switch abruptly between thousands of states, and the transition matrix of the model is structured to generate a high degree of volatility persistence. An empirical study on six financial time series shows that the FHMV process compares favorably to state-of-the-art volatility models in terms of in-sample fit and out-of-sample forecasting performance over time horizons ranging from 1 to 100 days. Supplementary materials for this article are available online.     

THE BIAS IN PERIODOGRAM ORDINATES AND THE ESTIMATION OF ARMA MODELS IN THE FREQUENCY DOMAIN

The paper has its origin in the finding that the frequency-domain estimation of ARh4A models can produce estimates which may be remarkably biased. Both of the frequency-domain estimation methods considered in the paper are based on the frequency-domain likelihood function, which depends on the periodogram ordinates of the time series. It is found that, as estimates of the spectrum ordinates, the corresponding periodogram ordinates may contain a rather remarkable bias, which again causes bias in the estimates of parameters produced by a frequency-domain estimation method of an ARMA model. The bias is reduced by tapering the observed time series. An example is given of estimation experiments for simulated time series from a pure autoregressive process of order two.     

R routines for performing estimation and statistical process control under copula-based time series models

Modeling serial dependence in time series is an important step in statistical process control. We provide a set of automatic routines useful for simulating and analyzing time series under a copula-based serial dependence. First, we introduce routines that generate time series data under a given copula. Second, we provide fully automated routines for obtaining maximum likelihood estimates for given time series data and then drawing a Shewhart-type control chart. Finally, real data are analyzed for illustration. We make the routines available as “Copula.Markov” package in R.     

Flowgraph Models for Generalized Phase Type Distributions Having Non-Exponential Waiting Times

ABSTRACT. Aalen (1995) introduced phase type distributions based on Markov processes for modelling disease progression in survival analysis. For tractability and to maintain the Markov property, these use exponential waiting times for transitions between states. This article extends the work of Aalen (1995) by generalizing these models to semi-Markov processes with non-exponential waiting times. The generalization allows more realistic modelling of the stages of a disease where the Markov property and exponential waiting times may not hold. Flowgraph models are introduced to provide a closed form for the distributions in situations involving non-exponential waiting times. Flowgraph models work where traditional methods of stochastic processes are intractable. Saddlepoint approximations are used in the analysis. Together, generalized phase type distributions, flowgraphs, and saddlepoint approximations create exciting and innovative prospects for the analysis of survival data.     

Statistical inference for discretely observed Markov jump processes

Summary.  Likelihood inference for discretely observed Markov jump processes with finite state space is investigated. The existence and uniqueness of the maximum likelihood estimator of the intensity matrix are investigated. This topic is closely related to the imbedding problem for Markov chains. It is demonstrated that the maximum likelihood estimator can be found either by the EM algorithm or by a Markov chain Monte Carlo procedure. When the maximum likelihood estimator does not exist, an estimator can be obtained by using a penalized likelihood function or by the Markov chain Monte Carlo procedure with a suitable prior. The methodology and its implementation are illustrated by examples and simulation studies.     

Structured priors for multivariate time series

A class of prior distributions for multivariate autoregressive models is presented. This class of priors is built taking into account the latent component structure that characterizes a collection of autoregressive processes. In particular, the state-space representation of a vector autoregressive process leads to the decomposition of each time series in the multivariate process into simple underlying components. These components may have a common structure across the series. A key feature of the proposed priors is that they allow the modeling of such common structure. This approach also takes into account the uncertainty in the number of latent processes, consequently handling model order uncertainty in the multivariate autoregressive framework. Posterior inference is achieved via standard Markov chain Monte Carlo (MCMC) methods. Issues related to inference and exploration of the posterior distribution are discussed. We illustrate the methodology analyzing two data sets: a synthetic data set with quasi-periodic latent structure, and seasonally adjusted US monthly housing data consisting of housing starts and housing sales over the period 1965 to 1974.