Markov Systems in a General State Space

In this article, we introduce and study Markov systems on general spaces (MSGS) as a first step of an entire theory on the subject. Also, all the concepts and basic results needed for this scope are given and analyzed. This could be thought of as an extension of the theory of a non homogeneous Markov system (NHMS) and that of a non homogeneous semi-Markov system on countable spaces, which has realized an interesting growth in the last thirty years. In addition, we study the asymptotic behaviour or ergodicity of Markov systems on general state spaces. The problem of asymptotic behaviour of Markov chains has been central for finite or countable spaces since the foundation of the subject. It has also been basic in the theory of NHMS and NHSMS. Two basic theorems are provided in answering the important problem of the asymptotic distribution of the population of the memberships of a Markov system that lives in the general space (X, ?(X)). Finally, we study the total variability from the invariant measure of the Markov system given that there exists an asymptotic behaviour. We prove a theorem which states that the total variation is finite. This problem is known also as the coupling problem.     

Markov Chain Monte Carlo Methods for Fitting Spatiotemporal Stochastic Models in Plant Epidemiology

Strategies for controlling plant epidemics are investigated by fitting continuous time spatiotemporal stochastic models to data consisting of maps of disease incidence observed at discrete times. Markov chain Monte Carlo methods are used for fitting two such models to data describing the spread of citrus tristeza virus (CTV) in an orchard. The approach overcomes some of the difficulties encountered when fitting stochastic models to infrequent observations of a continuous process. The results of the analysis cast doubt on the effectiveness of a strategy identified from a previous spatial analysis of the CTV data. Extensions of the approaches to more general models and other problems are also considered.     

Conditional Likelihood Estimators for Hidden Markov Models and Stochastic Volatility Models

ABSTRACT.  This paper develops a new contrast process for parametric inference of general hidden Markov models, when the hidden chain has a non-compact state space. This contrast is based on the conditional likelihood approach, often used for ARCH-type models. We prove the strong consistency of the conditional likelihood estimators under appropriate conditions. The method is applied to the Kalman filter (for which this contrast and the exact likelihood lead to asymptotically equivalent estimators) and to the discretely observed stochastic volatility models.     

On nonlinear models for time series

The paper is a review of nonlinear processes used in time series analysis and presents some new original results about stationary distribution of a nonlinear autoregres-sive process of the first order. The following models are considered: nonlinear autoregessive processes, threshold AR processes, threshold MA processes, bilinear models, auto-regressive models with random parameters including double stochastic models, exponential AR models, generalized threshold models and smooth transition autoregressive models, Some tests for linearity of processes are also presented.     

Spatiotemporal prediction for log-Gaussian Cox processes

Space–time point pattern data have become more widely available as a result of technological developments in areas such as geographic information systems. We describe a flexible class of space–time point processes. Our models are Cox processes whose stochastic intensity is a space–time Ornstein–Uhlenbeck process. We develop moment-based methods of parameter estimation, show how to predict the underlying intensity by using a Markov chain Monte Carlo approach and illustrate the performance of our methods on a synthetic data set.     

Choice between Semi-parametric Estimators of Markov and Non-Markov Multi-state Models from Coarsened Observations

Abstract.  We consider models based on multivariate counting processes, including multi-state models. These models are specified semi-parametrically by a set of functions and real parameters. We consider inference for these models based on coarsened observations, focusing on families of smooth estimators such as produced by penalized likelihood. An important issue is the choice of model structure, for instance, the choice between a Markov and some non-Markov models. We define in a general context the expected Kullback–Leibler criterion and we show that the likelihood-based cross-validation (LCV) is a nearly unbiased estimator of it. We give a general form of an approximate of the leave-one-out LCV. The approach is studied by simulations, and it is illustrated by estimating a Markov and two semi-Markov illness–death models with application on dementia using data of a large cohort study.     

Flexible Latent‐State Modelling of Old Faithful's Eruption Inter‐Arrival Times in 2009

This paper is concerned with the analysis of a time series comprising the eruption inter‐arrival times of the Old Faithful geyser in 2009. The series is much longer than other well‐documented ones and thus gives a more comprehensive insight into the dynamics of the geyser. Basic hidden Markov models with gamma state‐dependent distributions and several extensions are implemented. In order to better capture the stochastic dynamics exhibited by Old Faithful, the different non‐standard models under consideration seek to increase the flexibility of the basic models in various ways: (i) by allowing non‐geometric distributions for the times spent in the different states; (ii) by increasing the memory of the underlying Markov chain, with or without assuming additional structure implied by mixture transition distribution models; and (iii) by incorporating feedback from the observation process on the latent process. In each case it is shown how the likelihood can be formulated as a matrix product which can be conveniently maximized numerically.     

Goodness-of-fit Tests for Semi-Markov and Markov Survival Models with One Intermediate State

Survival data with one intermediate state are described by semi-Markov and Markov models for counting processes whose intensities are defined in terms of two stopping times T ₁< T ₂. Problems of goodness-of-fit for these models are studied. The test statistics are proposed by comparing Nelson–Aalen estimators for data stratified according to T ₁. Asymptotic distributions of these statistics are established in terms of the weak convergence of some random fields. Asymptotic consistency of these test statistics is also established. Simulation studies are included to indicate their numerical performance.     

Maximum likelihood from spatial random effects models via the stochastic approximation expectation maximization algorithm

We introduce a class of spatial random effects models that have Markov random fields (MRF) as latent processes. Calculating the maximum likelihood estimates of unknown parameters in SREs is extremely difficult, because the normalizing factors of MRFs and additional integrations from unobserved random effects are computationally prohibitive. We propose a stochastic approximation expectation-maximization (SAEM) algorithm to maximize the likelihood functions of spatial random effects models. The SAEM algorithm integrates recent improvements in stochastic approximation algorithms; it also includes components of the Newton-Raphson algorithm and the expectation-maximization (EM) gradient algorithm. The convergence of the SAEM algorithm is guaranteed under some mild conditions. We apply the SAEM algorithm to three examples that are representative of real-world applications: a state space model, a noisy Ising model, and segmenting magnetic resonance images (MRI) of the human brain. The SAEM algorithm gives satisfactory results in finding the maximum likelihood estimate of spatial random effects models in each of these instances.     

HEAVY-TAILED-DISTRIBUTED THRESHOLD STOCHASTIC VOLATILITY MODELS IN FINANCIAL TIME SERIES

To capture mean and variance asymmetries and time‐varying volatility in financial time series, we generalize the threshold stochastic volatility (THSV) model and incorporate a heavy‐tailed error distribution. Unlike existing stochastic volatility models, this model simultaneously accounts for uncertainty in the unobserved threshold value and in the time‐delay parameter. Self‐exciting and exogenous threshold variables are considered to investigate the impact of a number of market news variables on volatility changes. Adopting a Bayesian approach, we use Markov chain Monte Carlo methods to estimate all unknown parameters and latent variables. A simulation experiment demonstrates good estimation performance for reasonable sample sizes. In a study of two international financial market indices, we consider two variants of the generalized THSV model, with US market news as the threshold variable. Finally, we compare models using Bayesian forecasting in a value‐at‐risk (VaR) study. The results show that our proposed model can generate more accurate VaR forecasts than can standard models.     

A Class of Non-Gaussian State Space Models With Exact Likelihood Inference

The likelihood function of a general nonlinear, non-Gaussian state space model is a high-dimensional integral with no closed-form solution. In this article, I show how to calculate the likelihood function exactly for a large class of non-Gaussian state space models that include stochastic intensity, stochastic volatility, and stochastic duration models among others. The state variables in this class follow a nonnegative stochastic process that is popular in econometrics for modeling volatility and intensities. In addition to calculating the likelihood, I also show how to perform filtering and smoothing to estimate the latent variables in the model. The procedures in this article can be used for either Bayesian or frequentist estimation of the model’s unknown parameters as well as the latent state variables. Supplementary materials for this article are available online.     

Transient analysis of a finite source discrete-time queueing system using homogeneous Markov system with state size capacities (HMS/c)

Abstract

In this article, a finite source discrete-time queueing system is modeled as a discrete-time homogeneous Markov system with finite state size capacities (HMS/c) and transition priorities. This Markov system is comprised of three states. The first state of the HMS/c corresponds to the source and the second one to the state with the servers. The second state has a finite capacity which corresponds to the number of servers. The members of the system which can not enter the second state, due to its finite capacity, enter the third state which represents the system's queue. In order to examine the variability of the state sizes recursive formulae for their factorial and mixed factorial moments are derived in matrix form. As a consequence the probability mass function of each state size can be evaluated. Also the expected time in queue is computed by means of the interval transition probabilities. The theoretical results are illustrated by a numerical example.     

Modelling of unexpected shift in SPC

Optimal statistical process control (SPC) requires models of both in-control and out-of-control process states. Whereas a normal distribution is the generally accepted model for the in-control state, there is a doubt as to the existence of reliable models for out-of-control cases. Various process models, available in the literature, for discrete manufacturing systems (parts industry) can be treated as bounded discrete-space Markov chains, completely characterized by the original in-control state and a transition matrix for shifts to an out-of-control state. The present work extends these models by using a continuous-state Markov chain, incorporating non-random corrective actions. These actions are to be realized according to the SPC technique and should substantially affect the model. The developed stochastic model yields a Laplace distribution of a process mean. An alternative approach, based on the Information theory, also results in a Laplace distribution. Real-data tests confirm the applicability of a Laplace distribution for the parts industry and show that the distribution parameter is mainly controlled by the SPC sample size.     

Reliabilities of a single-unit system with multi-phased missions

ABSTRACT

Phased-mission systems (PMS) can be widely found in a lot of practical application areas. Reliability evaluations and analysis for this kind of systems become important issues. The reliability of PMS is typically defined as the probability that the system successfully accomplishes the missions of all phases. However, the k-out-of-n system success criterion for PMS has not been investigated. In this paper, according to this criterion, we develop two new models, which are static and dynamic, respectively. The assumptions for these two models are described in detail as well. The system reliabilities for both models are presented for the first time by employing finite Markov chain imbedding approach (FMCIA). In terms of FMCIA, we define different state spaces for the two models, and transition probability matrices are obtained. Then some numerical examples are given to illustrate the application of FMCIA. Finally, some discussions are made and conclusions are summarized.     

Forecasting interest rates volatilities by GARCH (1,1) and stochastic volatility models

In this paper, we compare the forecast ability of GARCH(1,1) and stochastic volatility models for interest rates. The stochastic volatility is estimated using Markov chain Monte Carlo methods. The comparison is based on daily data from 1994 to 1996 for the ten year swap rates for Deutsch Mark, Japanese Yen, and Pound Sterling. Various forecast horizons are considered. It turns out that forecasts based on stochastic volatility models are in most cases superiour to those obtained by GARCH(1,1) models.     

Stochastic models of geriatric patient bed occupancy behaviour

This paper considers a census approach to the modelling of the time that geriatric patients spend in hospital and subsequently in the community by using a stochastic compartmental Markov model. A selection process is developed using maximum likelihood estimation to fit the model to daily census data on the duration spent in the hospital or the community. Census data are used as they are easy to collect and therefore maximize the usability of the model. The model is fitted to an extensive 16-year data-set and shown to provide realistic estimates of movements of patients by using only a single day's census result.     

Extensions of Fill's algorithm for perfect simulation

Fill's algorithm for perfect simulation for attractive finite state space models, unbiased for user impatience, is presented in terms of stochastic recursive sequences and extended in two ways. Repulsive discrete Markov random fields with two coding sets like the auto-Poisson distribution on a lattice with 4-neighbourhood can be treated as monotone systems if a particular partial ordering and quasi-maximal and quasi-minimal states are used. Fill's algorithm then applies directly. Combining Fill's rejection sampling with sandwiching leads to a version of the algorithm which works for general discrete conditionally specified repulsive models. Extensions to other types of models are briefly discussed.     

Cross-infection in epidemics spread by carriers

A block-structured transient Markov process is introduced to describe an epidemic spreading within two linked populations, of carriers and susceptibles. The epidemic terminates as soon as there are no more carriers or susceptibles present in the population. Our purpose is to determine the distribution of the final susceptible and carrier states, and of any integral path for the susceptible process. The transient epidemic state is also briefly discussed. Then, the model is extended to allow the recovery of infected individuals. Finally, several particular models, some known, are used for illustration.     

A Baseline-free Procedure for Transformation Models Under Interval Censorship

An important property of Cox regression model is that the estimation of regression parameters using the partial likelihood procedure does not depend on its baseline survival function. We call such a procedure baseline-free. Using marginal likelihood, we show that an baseline-free procedure can be derived for a class of general transformation models under interval censoring framework. The baseline-free procedure results a simplified and stable computation algorithm for some complicated and important semiparametric models, such as frailty models and heteroscedastic hazard/rank regression models, where the estimation procedures so far available involve estimation of the infinite dimensional baseline function. A detailed computational algorithm using Markov Chain Monte Carlo stochastic approximation is presented. The proposed procedure is demonstrated through extensive simulation studies, showing the validity of asymptotic consistency and normality. We also illustrate the procedure with a real data set from a study of breast cancer. A heuristic argument showing that the score function is a mean zero martingale is provided.