Likelihood ratio test for testing order of continuous time finite markov chains

Anderson and Goodman ( 1957) have obtained the likelihood ratio tests and chi-square tests for testing the hypothesis about the order of discrete time finite Markov chains, On the similar lines we have obtained likeli¬hood ratio tests and chi-square tests (asymptotic) for testing hypotheses about the order of continuous time Markov chains (MC) with finite state space.     

A note on the passage time of finite-state Markov chains

Consider a Markov chain with finite state {0, 1, …, d}. We give the generation functions (or Laplace transforms) of absorbing (passage) time in the following two situations: (1) the absorbing time of state d when the chain starts from any state i and absorbing at state d; (2) the passage time of any state i when the chain starts from the stationary distribution supposed the chain is time reversible and ergodic. Example shows that it is more convenient compared with the existing methods, especially we can calculate the expectation of the absorbing time directly.     

Development of Specific hypothesis tests for estimated markov chains

We develop and evaluate the validity and power of two specific tests for the transition probabilities in a Markov chain estimated from aggregate frequency data. The two null hypotheses considered are (1) constancy of the diagonal elements of the one-step transition probability matrix and (2) an arbitrarily chosen transition probability’s being equal to a specific value. The formation of tests uses a general framework for statistical inference on estimated Markov processes; we also indicate how this framework can be used to form tests for a variety of other hypotheses. The validity and power performance of the two tests formed in this paper are examined in factorially designed Monte Carlo experiments. The results indicate that the proposed tests lead to type I error probabilities which are close to the desired levels and to high power against even small deviations from the null hypotheses considered.     

Minimal auxiliary Markov chains through sequential elimination of states

When using an auxiliary Markov chain to compute the distribution of a pattern statistic, the computational complexity is directly related to the number of Markov chain states. Theory related to minimal deterministic finite automata have been applied to large state spaces to reduce the number of Markov chain states so that only a minimal set remains. In this paper, a characterization of equivalent states is given so that extraneous states are deleted during the process of forming the state space, improving computational efficiency. The theory extends the applicability of Markov chain based methods for computing the distribution of pattern statistics.     

A bayesian method in determining the order of a finite state markov chain

In this paper, we use the Bayesian method in the application of hypothesis testing and model selection to determine the order of a Markov chain. The criteria used are based on Bayes factors with noninformative priors. Com¬parisons with the commonly used AIC and BIC criteria are made through an example and computer simulations. The results show that the proposed method is better than the AIC and BIC criteria, especially for Markov chains with higher orders and larger state spaces.     

A three-state continuous time Markov chain model for HIV disease burden

Plasma HIV viral load (VL) is the clinical indicator used to evaluate disease burden for HIV-infected patients. We developed a covariate-adjusted, three-state, homogenous continuous time Markov chain model for HIV/AIDS disease burden among subgroups. We defined Detectable and Undetectable HIV VL levels as two transient states and Death as the third absorbing state. We implemented the exact maximum likelihood method to estimate the parameters with related asymptotic distribution to conduct hypothesis testing. We evaluated the proposed model using HIV-infected individuals from South Carolina (SC) HIV surveillance data. Using the developed model, we estimated and compared the transition hazards, transition probabilities, and the state-specific duration for HIV-infected individuals. We examined gender, race/ethnicity, age, CD4 count, place of residence, and antiretroviral treatment regimen prescribed at the beginning of the study period. We found that patients with a higher CD4 count, increased age, heterosexual orientation, white, and single tablet regimen users were associated with reduced risk of transitioning to a Detectable VL from an Undetectable VL, whereas shorter time since diagnosis, being male, and injection drug use increased the risk of the same transition.     

Reversible jump, birth-and-death and more general continuous time Markov chain Monte Carlo samplers

Summary. Reversible jump methods are the most commonly used Markov chain Monte Carlo tool for exploring variable dimension statistical models. Recently, however, an alternative approach based on birth-and-death processes has been proposed by Stephens for mixtures of distributions. We show that the birth-and-death setting can be generalized to include other types of continuous time jumps like split-and-combine moves in the spirit of Richardson and Green. We illustrate these extensions both for mixtures of distributions and for hidden Markov models. We demonstrate the strong similarity of reversible jump and continuous time methodologies by showing that, on appropriate rescaling of time, the reversible jump chain converges to a limiting continuous time birth-and-death process. A numerical comparison in the setting of mixtures of distributions highlights this similarity.     

Computation of the stationary distribution of a markov chain

Eight algorithms are considered for the computation of the stationary distribution l´ of a finite Markov chain with associated probability transition matrix P. The recommended algorithm is based on solving l´(I—P+eú)=ú, where e is the column vector of ones and u´ is a row vector satisfying u´e ≠0.An error analysis is presented for any such u including the choices ú= e_jP and ú=e´_j where é_j is the jth row of the identity matrix. Computationalcomparisons between five of the algorithms are made based on twenty 8 x 8, twenty 20 x 20, and twenty 40 x 40 transition matrices. The matrix (I—P+eú)^?1 is shown to be a non-singular generalized inverse of I—P when the unit root of P is simple and úe ≠ 0. A simple closed form expression is obtained for the Moore-Penrose inverse of I—P whenI—P has nullity one     

Bounding convergence rates for Markov chains: An example of the use of computer algebra

Kolassa and Tanner (J. Am. Stat. Assoc. (1994) 89, 697–702) present the Gibbs-Skovgaard algorithm for approximate conditional inference. Kolassa (Ann Statist. (1999), 27, 129–142) gives conditions under which their Markov chain is known to converge. This paper calculates explicity bounds on convergence rates in terms calculable directly from chain transition operators. These results are useful in cases like those considered by Kolassa (1999).     

Quotas on runs of successes and failures in a multi-state markov chain

For the time-homogeneous multi-state Markov chain {X_n,n≧0} with states labeled as "0" (success) and "f"(failure), f=1,2,… the waiting time problems to be discussed arise by setting quotas on runs of success and failures. Some particular cases are considered.     

A structural model for the markov chain of change in vote intention

Assuming a first-order Markov chain, we propose a structural model for the transition probabilities in vote intention. The proposed model utilizes the ordering among the categories representing vote intentions and carries the flavor of distance models. It also allows a stochastic ordering among distributions reflecting the extent of change. The model is easy to fit and provides a nice interpretation of the data. The model is applied to a panel study of vote intention acquired through six successive interviews before the 1940 Presidential election in Erie County, Ohio.     

Estimating the order of a hidden markov model

While the estimation of the parameters of a hidden Markov model has been studied extensively, the consistent estimation of the number of hidden states is still an unsolved problem. The AIC and BIC methods are used most commonly, but their use in this context has not been justified theoretically. The author shows that for many common models, the penalized minimum‐distance method yields a consistent estimate of the number of hidden states in a stationary hidden Markov model. In addition to addressing the identifiability issues, she applies her method to a multiple sclerosis data set and assesses its performance via simulation.     

Exact filtering for partially observed continuous time models

Summary.  The forward–backward algorithm is an exact filtering algorithm which can efficiently calculate likelihoods, and which can be used to simulate from posterior distributions. Using a simple result which relates gamma random variables with different rates, we show how the forward–backward algorithm can be used to calculate the distribution of a sum of gamma random variables, and to simulate from their joint distribution given their sum. One application is to calculating the density of the time of a specific event in a Markov process, as this time is the sum of exponentially distributed interevent times. This enables us to apply the forward–backward algorithm to a range of new problems. We demonstrate our method on three problems: calculating likelihoods and simulating allele frequencies under a non-neutral population genetic model, analysing a stochastic epidemic model and simulating speciation times in phylogenetics.     

A markov regression model for the analysis of the postpartum lactational amenorrhea

Failure time data represent a particular case of binary longitudinal data. The corresponding analysis of the effect of explanatory covariates repeatedly collected over time on the failure rate has been largely facilitated by the Cox semi-parametric regression model. However, neither the interpretation of the estimated parameters associated with time-dependent covariates is straight-forward, nor does this model fully account for the dynamics of the effect of a covariate over time. Markovian regression models appear as complementary tools to address these specific issues from the predictive point of view. We illustrate these aspects using data from the WHO multicenter study, which was designed to analyze the relation between the duration of postpartum lactational amenorrhea and the breastfeeding pattern. One of the main advantage of this approach applied to the field of reproductive epidemiology was to provide a flexible tool, easily and directly understood by clinicians and fieldworkers, for simulating situations, which were still unobserved, and to predict their effects on the duration of amenorrhea.     

A a comparison of the markov chain and the integral equation approaches for evaluating the run length distribution of quality control charts

Two methods that are often used to evaluate the run length distribution of quality control charts are the Markov chain and integral equation approaches. Both methods have been used to evaluate the cumulative sum (CUSUM) charts and the exponentially weighted moving average (EWMA) control charts. The Markov chain approach involves "discretiz-ing" the possible values which can be plotted. Using properties of finite Markov chains, expressions for the distribution of the run length, and for the average run length (ARL), can be obtained. For the CUSUM and EWMA charts there exist integral equations whose solution gives the ARL. Approximate methods can then be used to solve the integral equation. In this article we show that if the product midpoint rule is used to approximate the integral in the integral equation, then both approaches yield the same approximations for the ARL. In addition we show that the recursive expressions for the probability functions are the same for the two approaches. These results establish the integral equation approach as preferable whenever an integral equation can be found     

On optimality of the horvitz-thompson estimator under a markov process model

For any varying probability sampling design the Horvitz-Thompson (1952) estimator is shown to be optimal within the class of all unbiased estimators of a finite population total under a Markov process model     

Some remarks on subdesigns of symmetric designs

Let D(υ, k, λ) be a symmetric design containing a symmetric design D₁(υ₁, k₁, λ₁) (k₁ < k) and let x = υ₁(k ? k₁)/(υ ? υ₁). We show that k ≥(k₁ ? x)² + λ If equality holds, D₁ is called a tight subdesign of D. In the special case, λ₁ = λ, the inequality reduces to that of R.C. Bose and S.S. Shrikhande and tight subdesigns then correspond to their notion of Baer subdesigns. The possibilities for (7upsi;, k, λ) designs having Baer subdesigns are investigated.     

On U-statistics and von Mises statistics for a special class of Markov chains

We derive the Berry-Esséen theorem with optimal convergence rate for U-statistics and von Mises statistics associated with a special class of Markov chains occuring in the theory of dependence with complete connections.