Fitting DNA sequences through log-linear modelling with linear constraints

For some discrete state series, such as DNA sequences, it can often be postulated that its probabilistic behaviour is given by a Markov chain. For making the decision on whether or not an uncharacterized piece of DNA is part of the coding region of a gene, under the Markovian assumption, there are two statistical tools that are essential to be considered: the hypothesis testing of the order in a Markov chain and the estimators of transition probabilities. In order to improve the traditional statistical procedures for both of them when stationarity assumption can be considered, a new version for understanding the homogeneity hypothesis is proposed so that log-linear modelling is applied for conditional independence jointly with homogeneity restrictions on the expected means of transition counts in the sequence. In addition we can consider a variety of test-statistics and estimators by using φ-divergence measures. As special case of them the well-known likelihood ratio test-statistics and maximum-likelihood estimators are obtained.     

Bivariate Semi-Markov Process for Counterparty Credit Risk

We consider the problem of constructing an appropriate multivariate model to study counterparty credit risk in the credit rating migration problem. For this financial problem different multivariate Markov chain models were proposed. However, the Markovian assumption may be inappropriate for the study of the dynamics of credit ratings, which typically show non Markovian-like behavior. In this article, we develop a semi-Markov approach to study the counterparty credit risk by defining a new multivariate semi-Markov chain model. Methods are given for computing the transition probabilities, reliability functions and the price of a risky Credit Default Swap.     

A stationarity test on Markov chain models based on marginal distribution

A stationarity test on Markov chain models is proposed in this paper. Most of the previous test procedures for the Markov chain models have been done based on the conditional probabilities of a transition matrix. The likelihood ratio and Pearson type chi-square tests have been used for testing stationarity and order of Markov chains. This paper uses the efficient score test, an extension of the test developed by Tsiatis (1980) [18], for testing the stationarity of Markov chain models based on the marginal distribution as obtained by Azzalini (1994) [2]. For testing the suitability of the proposed method, a numerical example of real life data and simulation studies for comparison with an alternative test procedure are given.     

A New Model for Multivariate Markov Chains

We propose a new model for multivariate Markov chains of order one or higher on the basis of the mixture transition distribution (MTD) model. We call it the MTD‐Probit. The proposed model presents two attractive features: it is completely free of constraints, thereby facilitating the estimation procedure, and it is more precise at estimating the transition probabilities of a multivariate or higher‐order Markov chain than the standard MTD model.     

A latent Markov model for detecting patterns of criminal activity

Summary.  The paper investigates the problem of determining patterns of criminal behaviour from official criminal histories, concentrating on the variety and type of offending convictions. The analysis is carried out on the basis of a multivariate latent Markov model which allows for discrete covariates affecting the initial and the transition probabilities of the latent process. We also show some simplifications which reduce the number of parameters substantially; we include a Rasch-like parameterization of the conditional distribution of the response variables given the latent process and a constraint of partial homogeneity of the latent Markov chain. For the maximum likelihood estimation of the model we outline an EM algorithm based on recursions known in the hidden Markov literature, which make the estimation feasible also when the number of time occasions is large. Through this model, we analyse the conviction histories of a cohort of offenders who were born in England and Wales in 1953. The final model identifies five latent classes and specifies common transition probabilities for males and females between 5-year age periods, but with different initial probabilities.     

Weibull and generalised exponential overdispersion models with an application to ozone air pollution

We consider the problem of estimating the mean and variance of the time between occurrences of an event of interest (inter-occurrences times) where some forms of dependence between two consecutive time intervals are allowed. Two basic density functions are taken into account. They are the Weibull and the generalised exponential density functions. In order to capture the dependence between two consecutive inter-occurrences times, we assume that either the shape and/or the scale parameters of the two density functions are given by auto-regressive models. The expressions for the mean and variance of the inter-occurrences times are presented. The models are applied to the ozone data from two regions of Mexico City. The estimation of the parameters is performed using a Bayesian point of view via Markov chain Monte Carlo (MCMC) methods.     

A two-state markov model

Markov chains are used to model binary urine test results. Taking advantage of the transition mechanism of Markov chains, missing observations can be incorporated in the analysis. Maximum likelihood estimates of transition probabilities are computed. Formulas for empirical Bayes procedures are given.     

Bayesian inference for dynamic social network data

We consider a continuous-time model for the evolution of social networks. A social network is here conceived as a (di-) graph on a set of vertices, representing actors, and the changes of interest are creation and disappearance over time of (arcs) edges in the graph. Hence we model a collection of random edge indicators that are not, in general, independent. We explicitly model the interdependencies between edge indicators that arise from interaction between social entities. A Markov chain is defined in terms of an embedded chain with holding times and transition probabilities. Data are observed at fixed points in time and hence we are not able to observe the embedded chain directly. Introducing a prior distribution for the parameters we may implement an MCMC algorithm for exploring the posterior distribution of the parameters by simulating the evolution of the embedded process between observations.     

Confidence Bands for Comparison of Transition Probabilities in a Markov Chain Model

We consider graphs, confidence procedures and tests that can be used to compare transition probabilities in a Markov chain model with intensities specified by a Cox proportional hazard model. Under assumptions of this model, the regression coefficients provide information about the relative risks of covariates in one–step transitions, however, they cannot in general be used to to assess whether or not the covariates have a beneficial or detrimental effect on the endpoint events. To alleviate this problem, we consider graphical tests based on confidence procedures for a generalized Q–Q plot and for the difference between transition probabilities. The procedures are illustrated using data of the International Bone Marrow Transplant Registry.     

Parallel discretization of the Markov chain approximation for the autoregressive moving average chart

Abstract

In the Markov chain model of an autoregressive moving average chart, the post-transition states of nonzero transition probabilities are distributed along one-dimensional lines of a constant gradient over the state space. By considering this characteristic, we propose discretizing the state space parallel to the gradient of these one-dimensional lines. We demonstrate that our method substantially reduces the computational cost of the Markov chain approximation for the average run length in two- and three-dimensional state spaces. Also, we investigate the effect of these one-dimensional lines on the computational cost. Lastly, we generalize our method to state spaces larger than three dimensions.     

Estimation and tests for a longitudinal regression model based on the Markov chain

In this paper, the dependence of transition probabilities on covariates and a test procedure for covariate dependent Markov models are examined. The nonparametric test for the role of waiting time proposed by Jones and Crowley [M. Jones, J. Crowley, Nonparametric tests of the Markov model for survival data Biometrika 79 (3) (1992) 513–522] has been extended here to transitions and reverse transitions. The limitation of the Jones and Crowley method is that it does not take account of other covariates that might have association with the probabilities of transition. A simple test procedure is proposed that can be employed for testing: (i) the significance of association between covariates and transition probabilities, and (ii) the impact of waiting time on the transition probabilities. The procedure is illustrated using panel data on hospitalization of the elderly population in the USA from the Health and Retirement Survey (HRS).     

A Bayesian Model for Markov Chains via Jeffrey's Prior

Abstract

This work deals with the problem of Bayesian estimation of the transition probabilities associated with multistate Markov chain. The model is based on the Jeffreys' noninformative prior. The Bayesian estimator is approximated by means of MCMC techniques. A numerical study by simulation is done in order to compare the Bayesian estimator with the maximum likelihood estimator.     

Quantifying uncertainty in transdimensional Markov chain Monte Carlo using discrete Markov models

Bayesian analysis often concerns an evaluation of models with different dimensionality as is necessary in, for example, model selection or mixture models. To facilitate this evaluation, transdimensional Markov chain Monte Carlo (MCMC) relies on sampling a discrete indexing variable to estimate the posterior model probabilities. However, little attention has been paid to the precision of these estimates. If only few switches occur between the models in the transdimensional MCMC output, precision may be low and assessment based on the assumption of independent samples misleading. Here, we propose a new method to estimate the precision based on the observed transition matrix of the model-indexing variable. Assuming a first-order Markov model, the method samples from the posterior of the stationary distribution. This allows assessment of the uncertainty in the estimated posterior model probabilities, model ranks, and Bayes factors. Moreover, the method provides an estimate for the effective sample size of the MCMC output. In two model selection examples, we show that the proposed approach provides a good assessment of the uncertainty associated with the estimated posterior model probabilities.

     

Parameter estimation for a special class of Markov chains

The followin paper is dedicated to a special class of stationary Markov chains. The transition probabilities are constructed from bivariate distribution functions of the Morgenstem-Type. These Markov chains are defined by their stationary distribution and a parameter a controlling the correlation between succeeding values of the chain. Relevant properties of the Markov chain are discussed. Some estimations of the parameter a are studied. The maximum likelihood estimator is compared with a simple estimator.     

BAYESIAN DETECTION AND ANALYSIS OF CHANGING TRANSITION MATRICES OF STATIONARY MARKOV CHAINS

This paper considers the detection of abrupt changes in the transition matrix of a Markov chain from a Bayesian viewpoint. It derives Bayes factors and posterior probabilities for unknown numbers of change‐points, as well as the positions of the change‐points, assuming non‐informative but proper priors on the parameters and fixed upper bound. The Markov chain Monte Carlo approach proposed by Chib in 1998 for estimating multiple change‐points models is adapted for the Markov chain model. It is especially useful when there are many possible change‐points. The method can be applied in a wide variety of disciplines and is particularly relevant in the social and behavioural sciences, for analysing the effects of events on the attitudes of people.     

A New lifetime model for multivariate survival data with a surviving fraction

In this paper we propose a new lifetime model for multivariate survival data with a surviving fraction. We develop this model assuming that there are m types of unobservable competing risks, where each risk is related to a time of the occurrence of an event of interest. We explore the use of Markov chain Monte Carlo methods to develop a Bayesian analysis for the proposed model. We also perform a simulation study in order to analyse the frequentist coverage probabilities of credible interval derived from posteriors. Our modelling is illustrated through a real data set.     

Development of Specific hypothesis tests for estimated markov chains

We develop and evaluate the validity and power of two specific tests for the transition probabilities in a Markov chain estimated from aggregate frequency data. The two null hypotheses considered are (1) constancy of the diagonal elements of the one-step transition probability matrix and (2) an arbitrarily chosen transition probability’s being equal to a specific value. The formation of tests uses a general framework for statistical inference on estimated Markov processes; we also indicate how this framework can be used to form tests for a variety of other hypotheses. The validity and power performance of the two tests formed in this paper are examined in factorially designed Monte Carlo experiments. The results indicate that the proposed tests lead to type I error probabilities which are close to the desired levels and to high power against even small deviations from the null hypotheses considered.     

Online change detection of Markov chains with unknown post-change transition probabilities

ABSTRACT

In this paper, we investigate the performance of cumulative sum (CUSUM) stopping rules for the online detection of unknown change point in a time homogeneous Markov chain. Under the condition that the post-change transition probabilities are unknown, we proposed two CUSUM type schemes for the detection. The first scheme is based on the maximum likelihood estimates of the post-change transition probabilities. This scheme is limited by its computation burden, which is mitigated by another scheme based on the reference transition probabilities selected from a prior known region. We give the bounds of the mean delay time and the mean time between false alarms to illustrate the effectiveness of the proposed schemes. The results of the simulation also demonstrate the feasibility of the proposed schemes.     

Nonparametric estimation of transition probabilities in a non-Markov illness–death model

In this paper we consider nonparametric estimation of transition probabilities for multi-state models. Specifically, we focus on the illness-death or disability model. The main novelty of the proposed estimators is that they do not rely on the Markov assumption, typically assumed to hold in a multi-state model. We investigate the asymptotic properties of the introduced estimators, such as their consistency and their convergence to a normal law. Simulations demonstrate that the new estimators may outperform Aalen–Johansen estimators (the classical nonparametric tool for estimating the transition probabilities) in non-Markov situation. An illustration through real data analysis is included.     

Stochastic monotonicity and comparability of Markov chains with block-monotone transition matrices and their applications to queueing systems

ABSTRACT

Motivated by various applications in queueing theory, this article is devoted to the stochastic monotonicity and comparability of Markov chains with block-monotone transition matrices. First, we introduce the notion of block-increasing convex order for probability vectors, and characterize the block-monotone matrices in the sense of the block-increasing order and block-increasing convex order. Second, we characterize the Markov chain with general transition matrix by martingale and provide a stochastic comparison of two block-monotone Markov chains under the two block-monotone orders. Third, the stochastic comparison results for the Markov chains corresponding to the discrete-time GI/G/1 queue with different service distributions under the two block-monotone orders are given, and the lower bound and upper bound of the Markov chain corresponding to the discrete-time GI/G/1 queue in the sense of the block-increasing convex order are found.