Modelling data observed irregularly over space and regularly in time

When the data has been collected regularly over time and irregularly over space, it is difficult to impose an explicit auto-regressive structure over the space as it is over time. We study a phenomenon on a number of fixed locations. On each location the process forms an auto-regressive time series. The second-order dependence over space is reflected by the covariance matrix of the noise process, which is ‘white’ in time but not over the space. We consider the asymptotic properties of our inference methods, when the number of recordings in time only tends to infinity.     

State space modeling of multiple time series

Time series methods offer the possibility of making accurate forecasts even when the underlying structural model is unknown, by replacing the structural restrictions needed to reduce sampling error and improve forecasts with restrictions determined from the data. While there has been considerable success with relatively simple univariate time series modeling procedures, the complex interrela- tionships possible with multiple series requite more powerful techniques.Based on the insights of linear systems theory, a multivariate state space methos for both stationary and nonstationary problems is described and related to ARMA models. The states or dynamic factors of the procedure are chosen to be robust in the presence of model misspecification, in constrast to ARMA models which lack this property. In addition, by treating th emidel choice as a formal approximation problem certain new optimal properties of the procedure with respect to specification are established; in particular, it is shown that no other model of equal or smaller order fits the observed autocovariance sequence any better in the sense of a Hankel norm. Finally, in the treatment of nonstationary series, a natural decomposition into long run and short run dynamics results in easily implemented two step procedures that use characteristics of the data to identify and model trend and cycle components that correspond to cointegration and error correction models. Applications include annualo U.S. GNP and money stock growth rates, monthly California beef prices and inventories, and monthly stock prices for large retailers.     

Diagnostics for dependence within time series extremes

Summary. The analysis of extreme values within a stationary time series entails various assumptions concerning its long- and short-range dependence. We present a range of new diagnostic tools for assessing whether these assumptions are appropriate and for identifying structure within extreme events. These tools are based on tail characteristics of joint survivor functions but can be implemented by using existing estimation methods for extremes of univariate independent and identically distributed variables. Our diagnostic aids are illustrated through theoretical examples, simulation studies and by application to rainfall and exchange rate data. On the basis of these diagnostics we can explain characteristics that are found in the observed extreme events of these series and also gain insight into the properties of events that are more extreme than those observed.     

Volatility model selection for extremes of financial time series

Although both widely used in the financial industry, there is quite often very little justification why GARCH or stochastic volatility is preferred over the other in practice. Most of the relevant literature focuses on the comparison of the fit of various volatility models to a particular data set, which sometimes may be inconclusive due to the statistical similarities of both processes. With an ever growing interest among the financial industry in the risk of extreme price movements, it is natural to consider the selection between both models from an extreme value perspective. By studying the dependence structure of the extreme values of a given series, we are able to clearly distinguish GARCH and stochastic volatility models and to test statistically which one better captures the observed tail behaviour. We illustrate the performance of the method using some stock market returns and find that different volatility models may give a better fit to the upper or lower tails.     

State space modeling of multiple time series: a comment

This paper provides guidance in choosing k₁ andk₂ of the double k-class (KK) estimator such that it will improve upon both the ordinary least squares (OLS) and Stein-rule (SR) estimators in predictive mean squared error (PMSE). Asymptotic bias and mean squared error (MSE) results are derived for nonnormal and other cases. A simulation compares the KK estimator with the OLS and SR estimators.     

Statistical inference for discretely observed Markov jump processes

Summary.  Likelihood inference for discretely observed Markov jump processes with finite state space is investigated. The existence and uniqueness of the maximum likelihood estimator of the intensity matrix are investigated. This topic is closely related to the imbedding problem for Markov chains. It is demonstrated that the maximum likelihood estimator can be found either by the EM algorithm or by a Markov chain Monte Carlo procedure. When the maximum likelihood estimator does not exist, an estimator can be obtained by using a penalized likelihood function or by the Markov chain Monte Carlo procedure with a suitable prior. The methodology and its implementation are illustrated by examples and simulation studies.     

Bayesian parameter inference for partially observed stopped processes

We consider Bayesian parameter inference associated to partially-observed stochastic processes that start from a set B ₀ and are stopped or killed at the first hitting time of a known set A. Such processes occur naturally within the context of a wide variety of applications. The associated posterior distributions are highly complex and posterior parameter inference requires the use of advanced Markov chain Monte Carlo (MCMC) techniques. Our approach uses a recently introduced simulation methodology, particle Markov chain Monte Carlo (PMCMC) (Andrieu et al. 2010), where sequential Monte Carlo (SMC) (Doucet et al. 2001; Liu 2001) approximations are embedded within MCMC. However, when the parameter of interest is fixed, standard SMC algorithms are not always appropriate for many stopped processes. In Chen et al. (2005), Del Moral (2004), the authors introduce SMC approximations of multi-level Feynman-Kac formulae, which can lead to more efficient algorithms. This is achieved by devising a sequence of sets from B ₀ to A and then performing the resampling step only when the samples of the process reach intermediate sets in the sequence. The choice of the intermediate sets is critical to the performance of such a scheme. In this paper, we demonstrate that multi-level SMC algorithms can be used as a proposal in PMCMC. In addition, we introduce a flexible strategy that adapts the sets for different parameter proposals. Our methodology is illustrated on the coalescent model with migration.     

Threshold selection in jump-discriminant filter for discretely observed jump processes

Threshold estimation is one of the useful techniques in the inference for jump-type stochastic processes from discrete observations. In this method, a jump-discriminant filter is used to infer the continuous part and the jump part separately. Although there are several choices for the filter, statistics constructed via filters are often sensitive to the choice. This paper presents some numerical procedures for selecting a suitable filter based on observations.     

Particle filtering for partially observed Gaussian state space models

Summary. Solving Bayesian estimation problems where the posterior distribution evolves over time through the accumulation of data has many applications for dynamic models. A large number of algorithms based on particle filtering methods, also known as sequential Monte Carlo algorithms, have recently been proposed to solve these problems. We propose a special particle filtering method which uses random mixtures of normal distributions to represent the posterior distributions of partially observed Gaussian state space models. This algorithm is based on a marginalization idea for improving efficiency and can lead to substantial gains over standard algorithms. It differs from previous algorithms which were only applicable to conditionally linear Gaussian state space models. Computer simulations are carried out to evaluate the performance of the proposed algorithm for dynamic tobit and probit models.     

Exact filtering for partially observed continuous time models

Summary.  The forward–backward algorithm is an exact filtering algorithm which can efficiently calculate likelihoods, and which can be used to simulate from posterior distributions. Using a simple result which relates gamma random variables with different rates, we show how the forward–backward algorithm can be used to calculate the distribution of a sum of gamma random variables, and to simulate from their joint distribution given their sum. One application is to calculating the density of the time of a specific event in a Markov process, as this time is the sum of exponentially distributed interevent times. This enables us to apply the forward–backward algorithm to a range of new problems. We demonstrate our method on three problems: calculating likelihoods and simulating allele frequencies under a non-neutral population genetic model, analysing a stochastic epidemic model and simulating speciation times in phylogenetics.     

Quasi- and pseudo-maximum likelihood estimators for discretely observed continuous-time Markov branching processes

This article deals with quasi- and pseudo-likelihood estimation for a class of continuous-time multi-type Markov branching processes observed at discrete points in time. “Conventional” and conditional estimation are discussed for both approaches. We compare their properties and identify situations where they lead to asymptotically equivalent estimators. Both approaches possess robustness properties, and coincide with maximum likelihood estimation in some cases. Quasi-likelihood functions involving only linear combinations of the data may be unable to estimate all model parameters. Remedial measures exist, including the resort either to non-linear functions of the data or to conditioning the moments on appropriate sigma-algebras. The method of pseudo-likelihood may also resolve this issue. We investigate the properties of these approaches in three examples: the pure birth process, the linear birth-and-death process, and a two-type process that generalizes the previous two examples. Simulations studies are conducted to evaluate performance in finite samples.     

Geometric stick-breaking processes for continuous-time Bayesian nonparametric modeling

We propose a new class of time dependent random probability measures and show how this can be used for Bayesian nonparametric inference in continuous time. By means of a nonparametric hierarchical model we define a random process with geometric stick-breaking representation and dependence structure induced via a one dimensional diffusion process of Wright-Fisher type. The sequence is shown to be a strongly stationary measure-valued process with continuous sample paths which, despite the simplicity of the weights structure, can be used for inferential purposes on the trajectory of a discretely observed continuous-time phenomenon. A simple estimation procedure is presented and illustrated with simulated and real financial data.     

The efficient estimation of tail probabilities for extremes of moving average processes using conditional simulation

This paper describes a conditional simulation technique which can be used to estimate probabilities associated with the distribution of the maximum of a real-valued process which can be written in the form of a moving average. The class of processes to which the technique applies includes non-stationary and spatial processes, and autoregressive processes. The technique is shown to achieve a considerable variance reduction compared with the obvious simulation-based estimator, particularly for estimating small upper-tail probabilities.     

Sampled autocovariance and autocorrelation results for linear time processes

We derive an exact formula for the covariance between the sampled autocovariances at any two lags for a finite time series realisation from a general stationary autoregressive moving average process. We indicate, through one particular example, how this result can be used to deduce analogous formulae for any nonstationary model of the ARUMA class, a generalisation of the ARIMA models. Such formulae then allow us to obtain approximate expressions for the convariances between all pairs of serial correlations for finite realisations from the ARUMA model. We also note that, in the limit as the series length n → ∞, our results for the ARMA class retrieve those of Bartlett (1946). Finally, we investigate an improvement to the approximation that is obtained by applying Bartlett's general asymptotic formula to finite series realisations. That such an improvement should exist can immediately be seen by consideration of out results for the simplest case of a white noise process. However, we deduce the final improved approapproximation, for general models, in two ways - from (corrected) results due to Davies and Newbold (1980), and by an alternative approach to theirs.     

Hierarchical modeling with gaussian processes

We formulate a hierarchical version of the Gaussian Process model. In particular, we assume there to be data on several units randomly drawn from the same population. For each unit, several responses are available that arise from a Gaussian Process model. The parameters characterizing the Gaussian Process model for the units are modeled to arise from normal or gamma distributions. Results for two simulations are given that compare the performance of the hierarchical and non-hierarchical models.     

Regression models for time-varying extremes

A common approach to modelling extreme data are to consider the distribution of the exceedance value over a high threshold. This approach is based on the distribution of excess, which follows the generalized Pareto distribution (GPD) and has shown to be adequate for this type of situation. As with all data involving analysis in time, excesses above a threshold may also vary and suffer from the influence of covariates. Thus, the GPD distribution can be modelled by entering the presence of these factors. This paper presents a new model for extreme values, where GPD parameters are written on the basis of a dynamic regression model. The estimation of the model parameters is made under the Bayesian paradigm, with sampling points via MCMC. As with environmental data, behaviour data are related to other factors such as time and covariates such as latitude and distance from the sea. Simulation studies have shown the efficiency and identifiability of the model, and applying real rain data from the state of Piaui, Brazil, shows the advantage in predicting and interpreting the model against other similar models proposed in the literature.