Bayesian Nonparametric Hidden Markov Models with application to the analysis of copy-number-variation in mammalian genomes

We consider the development of Bayesian Nonparametric methods for product partition models such as Hidden Markov Models and change point models. Our approach uses a Mixture of Dirichlet Process (MDP) model for the unknown sampling distribution (likelihood) for the observations arising in each state and a computationally efficient data augmentation scheme to aid inference. The method uses novel MCMC methodology which combines recent retrospective sampling methods with the use of slice sampler variables. The methodology is computationally efficient, both in terms of MCMC mixing properties, and robustness to the length of the time series being investigated. Moreover, the method is easy to implement requiring little or no user-interaction. We apply our methodology to the analysis of genomic copy number variation.     

DATA-DRIVEN NONPARAMETRIC SPECTRAL DENSITY ESTIMATORS FOR ECONOMIC TIME SERIES: A MONTE CARLO STUDY

Spectral analysis at frequencies other than zero plays an increasingly important role in econometrics. A number of alternative automated data-driven procedures for nonparametric spectral density estimation have been suggested in the literature, but little is known about their finite-sample accuracy. We compare five such procedures in terms of their mean-squared percentage error across frequencies. Our data generating processes (DGP) include autoregressive-moving average (ARMA) models, fractionally integrated ARMA models and nonparametric models based on 16 commonly used macroeconomic time series. We find that for both quarterly and monthly data the autoregressive sieve estimator is the most reliable method overall.     

Calculating the autocovariances and the likelihood for periodic V ARMA models

This paper concerns the autocovariance calculation and likelihood evaluation for periodic vector ARMA models (PV ARMA). Based on a state space representation of PV ARMA models, we derive an algorithm for computing the PV ARMA autocovariances. The proposed method computes the autocovariances for distinct seasons separately, thereby facilitating efficient calculation for models with a large period. As a result, the obtained autocovariance calculation procedure is exploited in a periodic Chandrasekhar-type filter to evaluate the exact likelihood for Gaussian PV ARMA series. Empirical evidence shows the superiority of the periodic Chandrasekhar algorithm for likelihood evaluation over the Kalman-based one.     

Time series analysis of categorical data using auto-odds ratio function

In this paper, we consider the auto-odds ratio function (AORF) as a measure of serial association for a stationary time series process of categorical data at two different time points. Numerical measures such as the autocorrelation function (ACF) have no meaningful interpretation, unless the time series data are numerical. Instead, we use the AORF as a measure of association to study the serial dependency of the categorical time series for both ordinal and nominal categories. Biswas and Song [Discrete-valued ARMA processes. Stat Probab Lett. 2009;79(17):1884–1889] provided some results on this measure for Pegram's operator-based AR(1) process with binary responses. Here, we extend this measure to more general set-ups, i.e. for AR(p) and MA(q) processes and for a general number of categories. We discuss how this method can effectively be used in parameter estimation and model selection. Following Weiß [Empirical measures of signed serial dependence in categorical time series. J Stat Comput Simul. 2011;81(4):411–429], we derive the large sample distribution of the estimator of the AORF under independent and identically distributed (iid) set-up. Some simulation results and two categorical data examples (one is ordinal and other nominal) are presented to illustrate the proposed method.     

A note on spectral estimation for ARMA processes

The standard frequency domain approximation to the Gaussian likelihood of a sample from an ARMA process is considered. The Newton-Raphson and Gauss-Newton numerical maximisation algorithms are evaluated for this approximate likelihood and the relationships between these algorithms and those of Akaike and Hannan explored. In particular it is shown that Hannan's method has certain computational advantages compared to the other spectral estimation methods considered     

AN EMPIRICAL LIKELIHOOD APPROACH FOR NON‐GAUSSIAN VECTOR STATIONARY PROCESSES AND ITS APPLICATION TO MINIMUM CONTRAST ESTIMATION

We develop the empirical likelihood approach for a class of vector‐valued, not necessarily Gaussian, stationary processes with unknown parameters. In time series analysis, it is known that the Whittle likelihood is one of the most fundamental tools with which to obtain a good estimator of unknown parameters, and that the score functions are asymptotically normal. Motivated by the Whittle likelihood, we apply the empirical likelihood approach to its derivative with respect to unknown parameters. We also consider the empirical likelihood approach to minimum contrast estimation based on a spectral disparity measure, and apply the approach to the derivative of the spectral disparity. This paper provides rigorous proofs on the convergence of our two empirical likelihood ratio statistics to sums of gamma distributions. Because the fitted spectral model may be different from the true spectral structure, the results enable us to construct confidence regions for various important time series parameters without assuming specified spectral structures and the Gaussianity of the process.     

Positivity conditions for stochastic state space modelling of time series

This short paper clarifies some aspects of the balancing method for state space modelling of observed time series. This method may fail to satisfy the so-called positive real condition for stochastic processes. We illustrate this by theoretical spectral analysis and also by simulating univariate ARMA (1,1) models.     

Inference for cluster point processes with over- or under-dispersed cluster sizes

Cluster point processes comprise a class of models that have been used for a wide range of applications. While several models have been studied for the probability density function of the offspring displacements and the parent point process, there are few examples of non-Poisson distributed cluster sizes. In this paper, we introduce a generalization of the Thomas process, which allows for the cluster sizes to have a variance that is greater or less than the expected value. We refer to this as the cluster sizes being over- and under-dispersed, respectively. To fit the model, we introduce minimum contrast methods and a Bayesian MCMC algorithm. These are evaluated in a simulation study. It is found that using the Bayesian MCMC method, we are in most cases able to detect over- and under-dispersion in the cluster sizes. We use the MCMC method to fit the model to nerve fiber data, and contrast the results to those of a fitted Thomas process.

     

Positivity conditions for stochastic state space modelling of time series

This short paper clarifies some aspects of the balancing method for state space modelling of observed time series. This method may fail to satisfy the so-called positive real condition for stochastic processes. We illustrate this by theoretical spectral analysis and also by simulating univariate ARMA (1,1) models.     

Integer autoregressive models with structural breaks

Even though integer-valued time series are common in practice, the methods for their analysis have been developed only in recent past. Several models for stationary processes with discrete marginal distributions have been proposed in the literature. Such processes assume the parameters of the model to remain constant throughout the time period. However, this need not be true in practice. In this paper, we introduce non-stationary integer-valued autoregressive (INAR) models with structural breaks to model a situation, where the parameters of the INAR process do not remain constant over time. Such models are useful while modelling count data time series with structural breaks. The Bayesian and Markov Chain Monte Carlo (MCMC) procedures for the estimation of the parameters and break points of such models are discussed. We illustrate the model and estimation procedure with the help of a simulation study. The proposed model is applied to the two real biometrical data sets.     

A comparison of Bayesian model selection based on MCMC with an application to GARCH-type models

This paper presents a comprehensive review and comparison of five computational methods for Bayesian model selection, based on MCMC simulations from posterior model parameter distributions. We apply these methods to a well-known and important class of models in financial time series analysis, namely GARCH and GARCH-t models for conditional return distributions (assuming normal and t-distributions). We compare their performance with the more common maximum likelihood-based model selection for simulated and real market data. All five MCMC methods proved reliable in the simulation study, although differing in their computational demands. Results on simulated data also show that for large degrees of freedom (where the t-distribution becomes more similar to a normal one), Bayesian model selection results in better decisions in favor of the true model than maximum likelihood. Results on market data show the instability of the harmonic mean estimator and reliability of the advanced model selection methods.     

Analyses of infectious disease data from household outbreaks by Markov chain Monte Carlo methods

The analysis of infectious disease data presents challenges arising from the dependence in the data and the fact that only part of the transmission process is observable. These difficulties are usually overcome by making simplifying assumptions. The paper explores the use of Markov chain Monte Carlo (MCMC) methods for the analysis of infectious disease data, with the hope that they will permit analyses to be made under more realistic assumptions. Two important kinds of data sets are considered, containing temporal and non-temporal information, from outbreaks of measles and influenza. Stochastic epidemic models are used to describe the processes that generate the data. MCMC methods are then employed to perform inference in a Bayesian context for the model parameters. The MCMC methods used include standard algorithms, such as the Metropolis–Hastings algorithm and the Gibbs sampler, as well as a new method that involves likelihood approximation. It is found that standard algorithms perform well in some situations but can exhibit serious convergence difficulties in others. The inferences that we obtain are in broad agreement with estimates obtained by other methods where they are available. However, we can also provide inferences for parameters which have not been reported in previous analyses.     

Estimating models based on Markov jump processes given fragmented observation series

We consider the problem of estimating the rate matrix governing a finite-state Markov jump process given a number of fragmented time series. We propose to concatenate the observed series and to employ the emerging non-Markov process for estimation. We describe the bias arising if standard methods for Markov processes are used for the concatenated process, and provide a post-processing method to correct for this bias. This method applies to discrete-time Markov chains and to more general models based on Markov jump processes where the underlying state process is not observed directly. This is demonstrated in detail for a Markov switching model. We provide applications to simulated time series and to financial market data, where estimators resulting from maximum likelihood methods and Markov chain Monte Carlo sampling are improved using the presented correction.     

Monte Carlo approximation of likelihood function in spatial GLMMs through an empirical Bayes method

In spatial generalized linear mixed models (SGLMMs), statistical inference encounters problems, since random effects in the model imply high-dimensional integrals to calculate the marginal likelihood function. In this article, we temporarily treat parameters as random variables and express the marginal likelihood function as a posterior expectation. Hence, the marginal likelihood function is approximated using the obtained samples from the posterior density of the latent variables and parameters given the data. However, in this setting, misspecification of prior distribution of correlation function parameter and problems associated with convergence of Markov chain Monte Carlo (MCMC) methods could have an unpleasant influence on the likelihood approximation. To avoid these challenges, we utilize an empirical Bayes approach to estimate prior hyperparameters. We also use a computationally efficient hybrid algorithm by combining inverse Bayes formula (IBF) and Gibbs sampler procedures. A simulation study is conducted to assess the performance of our method. Finally, we illustrate the method applying a dataset of standard penetration test of soil in an area in south of Iran.     

Parameter Estimation for a Bidimensional Partially Observed Ornstein–Uhlenbeck Process with Biological Application

Abstract. We consider a bidimensional Ornstein–Uhlenbeck process to describe the tissue microvascularization in anti‐cancer therapy. Data are discrete, partial and noisy observations of this stochastic differential equation (SDE). Our aim is to estimate the SDE parameters. We use the main advantage of a one‐dimensional observation to obtain an easy way to compute the exact likelihood using the Kalman filter recursion, which allows to implement an easy numerical maximization of the likelihood. Furthermore, we establish the link between the observations and an ARMA process and we deduce the asymptotic properties of the maximum likelihood estimator. We show that this ARMA property can be generalized to a higher dimensional underlying Ornstein–Uhlenbeck diffusion. We compare this estimator with the one obtained by the well‐known expectation maximization algorithm on simulated data. Our estimation methods can be directly applied to other biological contexts such as drug pharmacokinetics or hormone secretions.     

Asymptotic Inefficiency of Mean-Correction on Parameter Estimation for a Periodic First-Order Autoregressive Model

A common practice in time series analysis is to fit a centered model to the mean-corrected data set. For stationary autoregressive moving-average (ARMA) processes, as far as the parameter estimation is concerned, fitting an ARMA model without intercepts to the mean-corrected series is asymptotically equivalent to fitting an ARMA model with intercepts to the observed series. We show that, related to the parameter least squares estimation of periodic ARMA models, the second approach can be arbitrarily more efficient than the mean-corrected counterpart. This property is illustrated by means of a periodic first-order autoregressive model. The asymptotic variance of the estimators for both approaches is derived. Moreover, empirical experiments based on simulations investigate the finite sample properties of the estimators.     

Inference for Lévy‐Driven Stochastic Volatility Models via Adaptive Sequential Monte Carlo

Abstract. We investigate simulation methodology for Bayesian inference in Lévy‐driven stochastic volatility (SV) models. Typically, Bayesian inference from such models is performed using Markov chain Monte Carlo (MCMC); this is often a challenging task. Sequential Monte Carlo (SMC) samplers are methods that can improve over MCMC; however, there are many user‐set parameters to specify. We develop a fully automated SMC algorithm, which substantially improves over the standard MCMC methods in the literature. To illustrate our methodology, we look at a model comprised of a Heston model with an independent, additive, variance gamma process in the returns equation. The driving gamma process can capture the stylized behaviour of many financial time series and a discretized version, fit in a Bayesian manner, has been found to be very useful for modelling equity data. We demonstrate that it is possible to draw exact inference, in the sense of no time‐discretization error, from the Bayesian SV model.     

A class of observation-driven random coefficient INAR(1) processes based on negative binomial thinning

Integer-valued time series models and their applications have attracted a lot of attention over the last years. In this paper, we introduce a class of observation-driven random coefficient integer-valued autoregressive processes based on negative binomial thinning, where the autoregressive parameter depends on the observed values of the previous moment. Basic probability and statistics properties of the process are established. The unknown parameters are estimated by the conditional least squares and empirical likelihood methods. Specially, we consider three aspects of the empirical likelihood method: maximum empirical likelihood estimate, confidence region and EL test. The performance of the two estimation methods is compared through simulation studies. Finally, an application to a real data example is provided.     

Identifying the time of step change in the mean of autocorrelated processes

Control charts are used to detect changes in a process. Once a change is detected, knowledge of the change point would simplify the search for and identification of the special cause. Consequently, having an estimate of the process change point following a control chart signal would be useful to process analysts. Change-point methods for the uncorrelated process have been studied extensively in the literature; however, less attention has been given to change-point methods for autocorrelated processes. Autocorrelation is common in practice and is often modeled via the class of autoregressive moving average (ARMA) models. In this article, a maximum likelihood estimator for the time of step change in the mean of covariance-stationary processes that fall within the general ARMA framework is developed. The estimator is intended to be used as an “add-on” following a signal from a phase II control chart. Considering first-order pure and mixed ARMA processes, Monte Carlo simulation is used to evaluate the performance of the proposed change-point estimator across a range of step change magnitudes following a genuine signal from a control chart. Results indicate that the estimator provides process analysts with an accurate and useful estimate of the last sample obtained from the unchanged process. Additionally, results indicate that if a change-point estimator designed for the uncorrelated process is applied to an autocorrelated process, the performance of the estimator can suffer dramatically.