On Block Skip Free Transition Matrices and First Passage Times

Starting from an abstract setting which extends the property “skip free to the left” for transition matrices to a partition of the state space, we develop bounds for the mean hitting time of a Markov chain to an arbitrary subset from an arbitrary initial law. We apply our theory to the embedded Markov chains associated with the M/G/1 and the GI/M/1 queueing systems. We also illustrate its applicability with an asymptotic analysis of a non-reversible Markovian star queueing network with losses.     

Mixing of MCMC algorithms

We analyse MCMC chains focusing on how to find simulation parameters that give good mixing for discrete time, Harris ergodic Markov chains on a general state space X having invariant distribution π. The analysis uses an upper bound for the variance of the probability estimate. For each simulation parameter set, the bound is estimated from an MCMC chain using recurrence intervals. Recurrence intervals are a generalization of recurrence periods for discrete Markov chains. It is easy to compare the mixing properties for different simulation parameters. The paper gives general advice on how to improve the mixing of the MCMC chains and a new methodology for how to find an optimal acceptance rate for the Metropolis-Hastings algorithm. Several examples, both toy examples and large complex ones, illustrate how to apply the methodology in practice. We find that the optimal acceptance rate is smaller than the general recommendation in the literature in some of these examples.     

Strong laws of large numbers for the mth-order asymptotic odd–even Markov chains indexed by an m-rooted Cayley tree

In this article, we will study the strong laws of large numbers and asymptotic equipartition property (AEP) for mth-order asymptotic odd–even Markov chains indexed by an m-rooted Cayley tree. First, the definition of mth-order asymptotic odd–even Markov chains indexed by an m-rooted Cayley tree is introduced, then the strong limit theorem for this Markov chains is established. Next, the strong laws of large numbers for the frequencies of ordered couple of states for mth-order asymptotic odd–even Markov chains indexed by an m-rooted Cayley tree are obtained. Finally, we prove the AEP for this Markov chains.     

A class of small deviation theorems for the random variables associated with mth-order asymptotic circular Markov chains

ABSTRACT

In this article, we study a class of small deviation theorems for the random variables associated with mth-order asymptotic circular Markov chains. First, the definition of mth-order asymptotic circular Markov chain is introduced, then by applying the known results of the limit theorem for mth-order non homogeneous Markov chain, the small deviation theorem on the frequencies of occurrence of states for mth-order asymptotic circular Markov chains is established. Next, the strong law of large numbers and asymptotic equipartition property for this Markov chains are obtained. Finally, some results of mth-order nonhomogeneous Markov chains are given.     

Variance reduction for Markov chains with application to MCMC

In this paper, we propose a novel variance reduction approach for additive functionals of Markov chains based on minimization of an estimate for the asymptotic variance of these functionals over suitable classes of control variates. A distinctive feature of the proposed approach is its ability to significantly reduce the overall finite sample variance. This feature is theoretically demonstrated by means of a deep non-asymptotic analysis of a variance reduced functional as well as by a thorough simulation study. In particular, we apply our method to various MCMC Bayesian estimation problems where it favorably compares to the existing variance reduction approaches.     

The generalizations of strong law of large numbers for asymptotic even–odd Markov chains indexed by a homogeneous tree

Yang et al. (Yang et al., J. Math. Anal. Appl., 410 (2014), 179–189.) have obtained the strong law of large numbers and asymptotic equipartition property for the asymptotic even–odd Markov chains indexed by a homogeneous tree. In this article, we are going to study the strong law of large numbers and the asymptotic equipartition property for a class of non homogeneous Markov chains indexed by a homogeneous tree which are the generalizations of above results. We also provide an example showing that our generalizations are not trivial.     

Variance estimation in the central limit theorem for Markov chains

This article concerns the variance estimation in the central limit theorem for finite recurrent Markov chains. The associated variance is calculated in terms of the transition matrix of the Markov chain. We prove the equivalence of different matrix forms representing this variance. The maximum likelihood estimator for this variance is constructed and it is proved that it is strongly consistent and asymptotically normal. The main part of our analysis consists in presenting closed matrix forms for this new variance. Additionally, we prove the asymptotic equivalence between the empirical and the maximum likelihood estimation (MLE) for the stationary distribution.     

A distance-based diagnostic for trans-dimensional Markov chains

Over the last decade the use of trans-dimensional sampling algorithms has become endemic in the statistical literature. In spite of their application however, there are few reliable methods to assess whether the underlying Markov chains have reached their stationary distribution. In this article we present a distance-based method for the comparison of trans-dimensional Markov chain sample output for a broad class of models. This diagnostic will simultaneously assess deviations between and within chains. Illustration of the analysis of Markov chain sample-paths is presented in simulated examples and in two common modelling situations: a finite mixture analysis and a change-point problem.     

Regeneration-based bootstrap for Markov chains

We consider a bootstrap method for Markov chains where the original chain is broken into a (random) number of cycles based on an atom (regeneration point) and the bootstrap scheme resamples from these cycles. We investigate the asymptotic accuracy of this method for the case of a sum (or a sample mean) related to the Markov chain. Under some standard moment conditions, the method is shown to be at least as good as the normal approximation, and better (second-order accurate) in the case of nonlattice summands. We give three examples to illustrate the applicability of our results.     

Markov Systems in a General State Space

In this article, we introduce and study Markov systems on general spaces (MSGS) as a first step of an entire theory on the subject. Also, all the concepts and basic results needed for this scope are given and analyzed. This could be thought of as an extension of the theory of a non homogeneous Markov system (NHMS) and that of a non homogeneous semi-Markov system on countable spaces, which has realized an interesting growth in the last thirty years. In addition, we study the asymptotic behaviour or ergodicity of Markov systems on general state spaces. The problem of asymptotic behaviour of Markov chains has been central for finite or countable spaces since the foundation of the subject. It has also been basic in the theory of NHMS and NHSMS. Two basic theorems are provided in answering the important problem of the asymptotic distribution of the population of the memberships of a Markov system that lives in the general space (X, ?(X)). Finally, we study the total variability from the invariant measure of the Markov system given that there exists an asymptotic behaviour. We prove a theorem which states that the total variation is finite. This problem is known also as the coupling problem.     

Asymptotic equipartition property for second-order circular Markov chains indexed by a two-rooted Cayley tree

In this paper, we introduce a model of a second-order circular Markov chain indexed by a two-rooted Cayley tree and establish two strong law of large numbers and the asymptotic equipartition property (AEP) for circular second-order finite Markov chains indexed by this homogeneous tree. In the proof, we apply a limit property for a sequence of multi-variable functions of a non homogeneous Markov chain indexed by such tree. As a corollary, we obtain the strong law of large numbers and AEP about the second-order finite homogeneous Markov chain indexed by the two-rooted homogeneous tree.     

Strong Law of Large Numbers for Countable Asymptotic Circular Markov Chains

In this article, we introduce the notion of a countable asymptotic circular Markov chain, and prove a strong law of large numbers: as a corollary, we generalize a well-known version of the strong law of large numbers for nonhomogeneous Markov chains, and prove the Shannon-McMillan-Breiman theorem in this context, extending the result for the finite case.     

Pseudo-likelihood estimation for a class of spatial Markov chains

We consider a class of finite state, two-dimensional Markov chains which can produce a rich variety of patterns and whose simulation is very fast. A parameterization is chosen to make the process nearly spatially homogeneous. We use a form of pseudo-likelihood estimation which results in quick determination of estimate. Parameters associated with boundary cells are estimated separately. We derive the asymptotic distribution of the maximum pseudo-likelihood estimates and show that the usual form of the variance matrix has to be modified to take account of local dependence. Standard error calculations based on the modified asymptotic variance are supported by a simulation study. The procedure is applied to an eight-state permeability pattern from a section of hydrocarbon reservoir rock.     

Coordinate sampler: a non-reversible Gibbs-like MCMC sampler

We derive a novel non-reversible, continuous-time Markov chain Monte Carlo sampler, called Coordinate Sampler, based on a piecewise deterministic Markov process, which is a variant of the Zigzag sampler of Bierkens et al. (Ann Stat 47(3):1288–1320, 2019). In addition to providing a theoretical validation for this new simulation algorithm, we show that the Markov chain it induces exhibits geometrical ergodicity convergence, for distributions whose tails decay at least as fast as an exponential distribution and at most as fast as a Gaussian distribution. Several numerical examples highlight that our coordinate sampler is more efficient than the Zigzag sampler, in terms of effective sample size.     

Over-relaxation methods and coupled Markov chains for Monte Carlo simulation

This paper is concerned with improving the performance of certain Markov chain algorithms for Monte Carlo simulation. We propose a new algorithm for simulating from multivariate Gaussian densities. This algorithm combines ideas from coupled Markov chain methods and from an existing algorithm based only on over-relaxation. The rate of convergence of the proposed and existing algorithms can be measured in terms of the square of the spectral radius of certain matrices. We present examples in which the proposed algorithm converges faster than the existing algorithm and the Gibbs sampler. We also derive an expression for the asymptotic variance of any linear combination of the variables simulated by the proposed algorithm. We outline how the proposed algorithm can be extended to non-Gaussian densities.     

Likelihood ratio tests in linear mixed models with one variance component

Summary.  We consider the problem of testing null hypotheses that include restrictions on the variance component in a linear mixed model with one variance component and we derive the finite sample and asymptotic distribution of the likelihood ratio test and the restricted likelihood ratio test. The spectral representations of the likelihood ratio test and the restricted likelihood ratio test statistics are used as the basis of efficient simulation algorithms of their null distributions. The large sample χ ² mixture approximations using the usual asymptotic theory for a null hypothesis on the boundary of the parameter space have been shown to be poor in simulation studies. Our asymptotic calculations explain these empirical results. The theory of Self and Liang applies only to linear mixed models for which the data vector can be partitioned into a large number of independent and identically distributed subvectors. One-way analysis of variance and penalized splines models illustrate the results.     

Reducing Periodicity of Fuzzy Markov Chains Based on Simulation Using Halton Sequence

We first introduce fuzzy finite Markov chains and present some of their fundamental properties based on possibility theory. We also bring in a way to convert fuzzy Markov chains to classic Markov chains. In addition, we simulate fuzzy Markov chain using different sizes. It is observed that the most of fuzzy Markov chains not only do have an ergodic behavior, but also they are periodic. Finally, using Halton quasi-random sequence we generate some fuzzy Markov chains, which are compared with the ones generated by the RAND function of MATLAB. Therefore, we improve the periodicity behavior of fuzzy Markov chains.     

Asymptotic variances for the multiplicative interaction model

When modelling two-way analysis of variance interactions by a multiplicative term-[Formula] asymptotic variances and covariances are derived for the parameters p, yi and Sj using maximum likelihood theory. The asymptotic framework is defined by a2/K where K is the number of observations per combination of the two factors and a2 the common variance of the eijk values. The results can be applied when K = 1. Two Monte Carlo studies were carried out to check the validity of the formulae for small values of 02/K and to assess their usefulness when replacing the unknown parameters by their estimations. The formulae fit well but the confidence regions produced are too narrow if the interaction term is small. The procedure is illustrated with two examples.     

Inference using attributable risk for a 2×2 case–control study

Asymptotic variance plays an important role in the inference using interval estimate of attributable risk. This paper compares asymptotic variances of attributable risk estimate using the delta method and the Fisher information matrix for a 2×2 case–control study due to the practicality of applications. The expressions of these two asymptotic variance estimates are shown to be equivalent. Because asymptotic variance usually underestimates the standard error, the bootstrap standard error has also been utilized in constructing the interval estimates of attributable risk and compared with those using asymptotic estimates. A simulation study shows that the bootstrap interval estimate performs well in terms of coverage probability and confidence length. An exact test procedure for testing independence between the risk factor and the disease outcome using attributable risk is proposed and is justified for the use with real-life examples for a small-sample situation where inference using asymptotic variance may not be valid.     

Asymptotic Methods of Enumeration and Applications to Markov Chain Models

Abstract

The purpose of this article is to present analytic methods for determining the asymptotic behaviour of the coefficents of power series that can be applied to homogeneous discrete quasi death and birth processes. It turns that there are in principle only three types for the asymptotic behaviour. The process either converges to the stationary distribution or it can be approximated in terms of a reflected Brownian motion or by a Brownian motion. In terms of Markov chains these cases correspond to positive recurrence, to null recurrence, and to non recurrence. The same results hold for the continuous case, too.