Markov Chains and De-initializing Processes

We define a notion of de-initializing Markov chains. We prove that to analyse convergence of Markov chains to stationarity, it suffices to analyse convergence of a de-initializing chain. Applications are given to Markov chain Monte Carlo algorithms and to convergence diagnostics.     

Mixing of MCMC algorithms

We analyse MCMC chains focusing on how to find simulation parameters that give good mixing for discrete time, Harris ergodic Markov chains on a general state space X having invariant distribution π. The analysis uses an upper bound for the variance of the probability estimate. For each simulation parameter set, the bound is estimated from an MCMC chain using recurrence intervals. Recurrence intervals are a generalization of recurrence periods for discrete Markov chains. It is easy to compare the mixing properties for different simulation parameters. The paper gives general advice on how to improve the mixing of the MCMC chains and a new methodology for how to find an optimal acceptance rate for the Metropolis-Hastings algorithm. Several examples, both toy examples and large complex ones, illustrate how to apply the methodology in practice. We find that the optimal acceptance rate is smaller than the general recommendation in the literature in some of these examples.     

A distance-based diagnostic for trans-dimensional Markov chains

Over the last decade the use of trans-dimensional sampling algorithms has become endemic in the statistical literature. In spite of their application however, there are few reliable methods to assess whether the underlying Markov chains have reached their stationary distribution. In this article we present a distance-based method for the comparison of trans-dimensional Markov chain sample output for a broad class of models. This diagnostic will simultaneously assess deviations between and within chains. Illustration of the analysis of Markov chain sample-paths is presented in simulated examples and in two common modelling situations: a finite mixture analysis and a change-point problem.     

Asymptotic Methods of Enumeration and Applications to Markov Chain Models

Abstract

The purpose of this article is to present analytic methods for determining the asymptotic behaviour of the coefficents of power series that can be applied to homogeneous discrete quasi death and birth processes. It turns that there are in principle only three types for the asymptotic behaviour. The process either converges to the stationary distribution or it can be approximated in terms of a reflected Brownian motion or by a Brownian motion. In terms of Markov chains these cases correspond to positive recurrence, to null recurrence, and to non recurrence. The same results hold for the continuous case, too.     

Strong laws of large numbers for the mth-order asymptotic odd–even Markov chains indexed by an m-rooted Cayley tree

In this article, we will study the strong laws of large numbers and asymptotic equipartition property (AEP) for mth-order asymptotic odd–even Markov chains indexed by an m-rooted Cayley tree. First, the definition of mth-order asymptotic odd–even Markov chains indexed by an m-rooted Cayley tree is introduced, then the strong limit theorem for this Markov chains is established. Next, the strong laws of large numbers for the frequencies of ordered couple of states for mth-order asymptotic odd–even Markov chains indexed by an m-rooted Cayley tree are obtained. Finally, we prove the AEP for this Markov chains.     

Fitting Markov chain models to discrete state series such as DNA sequences

Discrete state series such as DNA sequences can often be modelled by Markov chains. The analysis of such series is discussed in the context of log-linear models. The data produce contingency tables with similar margins due to the dependence of the observations. However, despite the unusual structure of the tables, the analysis is equivalent to that for data from multinomial sampling. The reason why the standard number of degrees of freedom is correct is explained by using theoretical arguments and the asymptotic distribution of the deviance is verified empirically. Problems involved with fitting high order Markov chain models, such as reduced power and computational expense, are also discussed.     

Using Markov Chains for Non-perennial Daily Streamflow Data Generation

The use of Markov chains to simulate non-perennial streamflow data is considered. A non-perennial stream may be thought as having three states, namely zero flow, increasing flow and decreasing flow, for which a three-state Markov chain can be constructed. Alternatively, two two-state Markov chains can be used, the first of which represents the existence and non-existence of flow, whereas the second deals with the increment and decrement in the flow for periods with flow. Probabilistic relationships between the two alternatives are derived. Their performances in simulating the state of the stream are compared on the basis of data from two different geographical regions in Turkey. It is concluded that both alternatives are capable of simulating the state of the stream.     

Recurrence relations between product moments of order statistics

A general result for obtaining recurrence relations between product moments of order statistics is established and this result is used to determine the recurrence relations between product moments of some doubly truncated distributions. The examples considered are Weibull, exponential, Pareto, power function and Cauchy distributions.     

Likelihood ratio test for testing order of continuous time finite markov chains

Anderson and Goodman ( 1957) have obtained the likelihood ratio tests and chi-square tests for testing the hypothesis about the order of discrete time finite Markov chains, On the similar lines we have obtained likeli¬hood ratio tests and chi-square tests (asymptotic) for testing hypotheses about the order of continuous time Markov chains (MC) with finite state space.     

Order statistics from non-identical power function random variables

In this paper, we derive some recurrence relations satisfied by the single and the product moments of order statistics arising from n independent and non-identically distributed power function random variables. These recurrence relations will enable one to compute all the single and the product moments of all order statistics in a simple recursive manner. The results for the multiple-outlier model are deduced as special cases. The results are further generalized to the case of truncated power function random variables.     

Approximations for Multiple Scan Statistics on the Circle

Multiple scan statistic is usually used by epidemiologist to test the uniformity or clustering of data. In this article, we extend the work of Lin (1999) to give a general expression for the moments of multiple scan statistic on a circle, and use these moments to approximate its distribution using Markov chain and compound Poisson approximations proposed by Huffer and Lin (1997a) and Lin (1993). Numerical results are presented to evaluate the performance of these approximations.     

Some Estimates for the Rate of Convergence in the Multidimensional Central Limit Theorem for Homogeneous Markov Chains

This paper obtains some estimates for the rate of convergence in the multi-dimensional central limit theorem for vector-valued functions of a homogeneous Markov chain without assuming the finiteness of their absolute third moment. These estimates have a universal character and generalize the results that hold when the third moments are finite.     

Infinite horizon reflected backward stochastic differential equations with Markov chains

We first give the existence and uniqueness results for infinite horizon backward stochastic differential equations with Markov chains, taking advantage of the martingale representation theorem and fixed point principle. Then we prove the well-posedness results for infinite horizon reflected backward stochastic differential equations with Markov chains, by virtue of the Snell envelope theory and contraction mapping method. Comparison theorems for the above two kinds of equations are also obtained, via the linearization approach or properties of reflected backward stochastic differential equations, respectively.     

Statistical inference for discretely observed Markov jump processes

Summary.  Likelihood inference for discretely observed Markov jump processes with finite state space is investigated. The existence and uniqueness of the maximum likelihood estimator of the intensity matrix are investigated. This topic is closely related to the imbedding problem for Markov chains. It is demonstrated that the maximum likelihood estimator can be found either by the EM algorithm or by a Markov chain Monte Carlo procedure. When the maximum likelihood estimator does not exist, an estimator can be obtained by using a penalized likelihood function or by the Markov chain Monte Carlo procedure with a suitable prior. The methodology and its implementation are illustrated by examples and simulation studies.     

A bayesian method in determining the order of a finite state markov chain

In this paper, we use the Bayesian method in the application of hypothesis testing and model selection to determine the order of a Markov chain. The criteria used are based on Bayes factors with noninformative priors. Com¬parisons with the commonly used AIC and BIC criteria are made through an example and computer simulations. The results show that the proposed method is better than the AIC and BIC criteria, especially for Markov chains with higher orders and larger state spaces.     

Moments of order statistics from doubly truncated continuous distributions and characterizations

In this paper, recurrence relations from a general class of doubly truncated continuous distributions which are satisfied by single as well as product moments of order statistics are obtained. Recurrence relations from doubly truncated generalized Weibull, exponential, Raleigh and logistic distributions have been derived as special cases of our result, Some previous results for doubly truncated Weibull, standard exponential, power function and Burr type XII distributions are obtained as special cases. The general recurrence relation of single moments has been used in the case of the left and right truncation to characterize the Weibull, Burr type XII and Pareto distributions.     

Reducing Periodicity of Fuzzy Markov Chains Based on Simulation Using Halton Sequence

We first introduce fuzzy finite Markov chains and present some of their fundamental properties based on possibility theory. We also bring in a way to convert fuzzy Markov chains to classic Markov chains. In addition, we simulate fuzzy Markov chain using different sizes. It is observed that the most of fuzzy Markov chains not only do have an ergodic behavior, but also they are periodic. Finally, using Halton quasi-random sequence we generate some fuzzy Markov chains, which are compared with the ones generated by the RAND function of MATLAB. Therefore, we improve the periodicity behavior of fuzzy Markov chains.     

Some Results on Modeling Biological Sequences and Web Navigation with a Semi Markov Chain

In the article, the entrance probabilities and the probability distribution of the number of transitions to a state are studied to provide some answers to questions related to state occupancies for the semi Markov model. Biological sequences and Web navigation are two cases that initially seem to be different but to a certain extent they do have similarities. Two main aspects of word occurrences in biological sequences are: (a) where do they occur and (b) how many times do they occur. In Web navigation the similar questions are (a) when a node is visited and (b) how many times a node is visited. So, the theoretical results of this study are applied to model these two cases and derive distributions of word location or node occurrence and frequency of occurrences. Rewards/costs are included in the Web navigation model and analytic forms for the means, variances, and moments of total interval rewards/costs are provided.