Confidence Bands for Comparison of Transition Probabilities in a Markov Chain Model

We consider graphs, confidence procedures and tests that can be used to compare transition probabilities in a Markov chain model with intensities specified by a Cox proportional hazard model. Under assumptions of this model, the regression coefficients provide information about the relative risks of covariates in one–step transitions, however, they cannot in general be used to to assess whether or not the covariates have a beneficial or detrimental effect on the endpoint events. To alleviate this problem, we consider graphical tests based on confidence procedures for a generalized Q–Q plot and for the difference between transition probabilities. The procedures are illustrated using data of the International Bone Marrow Transplant Registry.     

Asymptotic theory for the Cox semi-Markov illness-death model

Irreversible illness-death models are used to model disease processes and in cancer studies to model disease recovery. In most applications, a Markov model is assumed for the multistate model. When there are covariates, a Cox (1972, J Roy Stat Soc Ser B 34:187–220) model is used to model the effect of covariates on each transition intensity. Andersen et al. (2000, Stat Med 19:587–599) proposed a Cox semi-Markov model for this problem. In this paper, we study the large sample theory for that model and provide the asymptotic variances of various probabilities of interest. A Monte Carlo study is conducted to investigate the robustness and efficiency of Markov/Semi-Markov estimators. A real data example from the PROVA (1991, Hepatology 14:1016–1024) trial is used to illustrate the theory.     

A stationarity test on Markov chain models based on marginal distribution

A stationarity test on Markov chain models is proposed in this paper. Most of the previous test procedures for the Markov chain models have been done based on the conditional probabilities of a transition matrix. The likelihood ratio and Pearson type chi-square tests have been used for testing stationarity and order of Markov chains. This paper uses the efficient score test, an extension of the test developed by Tsiatis (1980) [18], for testing the stationarity of Markov chain models based on the marginal distribution as obtained by Azzalini (1994) [2]. For testing the suitability of the proposed method, a numerical example of real life data and simulation studies for comparison with an alternative test procedure are given.     

A latent Markov model for detecting patterns of criminal activity

Summary.  The paper investigates the problem of determining patterns of criminal behaviour from official criminal histories, concentrating on the variety and type of offending convictions. The analysis is carried out on the basis of a multivariate latent Markov model which allows for discrete covariates affecting the initial and the transition probabilities of the latent process. We also show some simplifications which reduce the number of parameters substantially; we include a Rasch-like parameterization of the conditional distribution of the response variables given the latent process and a constraint of partial homogeneity of the latent Markov chain. For the maximum likelihood estimation of the model we outline an EM algorithm based on recursions known in the hidden Markov literature, which make the estimation feasible also when the number of time occasions is large. Through this model, we analyse the conviction histories of a cohort of offenders who were born in England and Wales in 1953. The final model identifies five latent classes and specifies common transition probabilities for males and females between 5-year age periods, but with different initial probabilities.     

Logistic regression and other discrete data models for serially correlated observations

Summary We consider the analysis of discrete serially correlated data in the presence of time dependent covariates. If the interest is to relate the covariates to the marginal distribution of the data, Markov chains are an obvious tool to consider, but their use is complicated by the fact that they are expressed in terms of transitional rather than marginal probabilities. We show how to parametrize the transition matrix in a suitable way so that interpretation is as desired. The focus is on binary and Poisson data, but the methodology can be adopted also with other discrete data distributions.     

Semi-parametric extended Poisson process models for count data

A general framework for the analysis of count data (with covariates) is proposed using formulations for the transition rates of a state-dependent birth process. The form for the transition rates incorporates covariates proportionally, with the residual distribution determined from a smooth non-parametric state-dependent form. Computation of the resulting probabilities is discussed, leading to model estimation using a penalized likelihood function. Two data sets are used as illustrative examples, one representing underdispersed Poisson-like data and the other overdispersed binomial-like data.     

Markov Poisson regression models for discrete time series. Part 1: Methodology

This paper proposes and investigates a class of Markov Poisson regression models in which Poisson rate functions of covariates are conditional on unobserved states which follow a finite-state Markov chain. Features of the proposed model, estimation, inference, bootstrap confidence intervals, model selection and other implementation issues are discussed. Monte Carlo studies suggest that the proposed estimation method is accurate and reliable for single- and multiple-subject time series data; the choice of starting probabilities for the Markov process has little eff ect on the parameter estimates; and penalized likelihood criteria are reliable for determining the number of states. Part 2 provides applications of the proposed model.     

Flexible semi-parametric regression of state occupational probabilities in a multistate model with right-censored data

Inference for the state occupation probabilities, given a set of baseline covariates, is an important problem in survival analysis and time to event multistate data. We introduce an inverse censoring probability re-weighted semi-parametric single index model based approach to estimate conditional state occupation probabilities of a given individual in a multistate model under right-censoring. Besides obtaining a temporal regression function, we also test the potential time varying effect of a baseline covariate on future state occupation. We show that the proposed technique has desirable finite sample performances and its performance is competitive when compared with three other existing approaches. We illustrate the proposed methodology using two different data sets. First, we re-examine a well-known data set dealing with leukemia patients undergoing bone marrow transplant with various state transitions. Our second illustration is based on data from a study involving functional status of a set of spinal cord injured patients undergoing a rehabilitation program.     

Application of a non-homogeneous Markov chain with seasonal transition probabilities to ozone data

In this work, we assume that the sequence recording whether or not an ozone exceedance of an environmental threshold has occurred in a given day is ruled by a non-homogeneous Markov chain of order one. In order to account for the possible presence of cycles in the empirical transition probabilities, a parametric form incorporating seasonal components is considered. Results show that even though some covariates (namely, relative humidity and temperature) are not included explicitly in the model, their influence is captured in the behavior of the transition probabilities. Parameters are estimated using the Bayesian point of view via Markov chain Monte Carlo algorithms. The model is applied to ozone data obtained from the monitoring network of Mexico City, Mexico. An analysis of how the methodology could be used as an aid in the decision-making is also given.     

Analyzing dependence in incidence of diabetes and heart problem using generalized bivariate geometric models with covariates

For analyzing incidence data on diabetes and health problems, the bivariate geometric probability distribution is a natural choice but remained unexplored largely due to lack of models linking covariates with the probabilities of bivariate incidence of correlated outcomes. In this paper, bivariate geometric models are proposed for two correlated incidence outcomes. The extended generalized linear models are developed to take into account covariate dependence of the bivariate probabilities of correlated incidence outcomes for diabetes and heart diseases for the elderly population. The estimation and test procedures are illustrated using the Health and Retirement Study data. Two models are shown in this paper, one based on conditional-marginal approach and the other one based on the joint probability distribution with an association parameter. The joint model with association parameter appears to be a very good choice for analyzing the covariate dependence of the joint incidence of diabetes and heart diseases. Bootstrapping is performed to measure the accuracy of estimates and the results indicate very small bias.     

A New Model for Multivariate Markov Chains

We propose a new model for multivariate Markov chains of order one or higher on the basis of the mixture transition distribution (MTD) model. We call it the MTD‐Probit. The proposed model presents two attractive features: it is completely free of constraints, thereby facilitating the estimation procedure, and it is more precise at estimating the transition probabilities of a multivariate or higher‐order Markov chain than the standard MTD model.     

An alternative method for evaluating stationarity in transition models

Transition models are an important framework that can be used to model longitudinal categorical data. A relevant issue in applying these models is the condition of stationarity, or homogeneity of transition probabilities over time. We propose two tests to assess stationarity in transition models: Wald and likelihood-ratio tests, which do not make use of transition probabilities, using only the estimated parameters of the models in contrast to the classical test available in the literature. In this paper, we present two motivating studies, with ordinal longitudinal data, to which proportional odds transition models are fitted and the two proposed tests are applied as well as the classical test. Additionally, their performances are assessed through simulation studies. The results show that the proposed tests have good performance, being better for control of type-I error and they present equivalent power functions asymptotically. Also, the correlations between the Wald, likelihood-ratio and the classical test statistics are positive and large, an indicator of general concordance. Additionally, both of the proposed tests are more flexible and can be applied in studies with qualitative and quantitative covariates.     

Trends in smoking cessation: a Markov model approach

Intervention trials such as studies on smoking cessation may observe multiple, discrete outcomes over time. When the outcome is binary, participant observations may alternate between two states over the course of the study. The generalized estimating equation (GEE) approach is commonly used to analyze binary, longitudinal data in the context of independent variables. However, the sequence of observations may be assumed to follow a Markov chain with stationary transition probabilities when observations are made at fixed time points. Participants favoring the transition to one particular state over the other would be evidence of a trend in the observations. Using a log-transformed trend parameter, the determinants of a trend in a binary, longitudinal study may be evaluated by maximizing the likelihood function. A new methodology is presented here to test for the presence and determinants of a trend in binary, longitudinal observations. Empirical studies are evaluated and comparisons are made with the GEE approach. Practical application of the proposed method is made to the data available from an intervention study on smoking cessation.     

Online change detection of Markov chains with unknown post-change transition probabilities

ABSTRACT

In this paper, we investigate the performance of cumulative sum (CUSUM) stopping rules for the online detection of unknown change point in a time homogeneous Markov chain. Under the condition that the post-change transition probabilities are unknown, we proposed two CUSUM type schemes for the detection. The first scheme is based on the maximum likelihood estimates of the post-change transition probabilities. This scheme is limited by its computation burden, which is mitigated by another scheme based on the reference transition probabilities selected from a prior known region. We give the bounds of the mean delay time and the mean time between false alarms to illustrate the effectiveness of the proposed schemes. The results of the simulation also demonstrate the feasibility of the proposed schemes.     

Estimation in a model for a semi-Markov process with covariates under right-censoring

A semi-Markov model with covariates is proposed for a multi-state process with a finite number of states such that the transition probabilities between the states and the distribution functions of the duration times between the occurrence of two states depend on a discrete covariate. The hazard rates for the time elapsed between two successive states depend on the covariate through a proportional hazards model involving a set of regression parameters, while the transition probabilities depend on the covariate in an unspecified way. We propose estimators for these parameters and for the cumulative hazard functions of the sojourn times. A difficulty comes from the fact that when a sojourn time in a state is right-censored, the next state is unknown. We prove that our estimators are consistent and asymptotically Gaussian under the model constraints.     

A three-state disease model with interval-censored data: Estimation and applications to AIDS and cancer

In presence of interval-censored data, we propose a general three-state disease model with covariates. Such data can arise, for example, in epidemiologic studies of infectious disease where both the times of infection and disease onset are not directly observed, or in cancer studies where the time of disease metastasis is known up to a specified interval. The proposed model allows the distributions of the transition times between states to depend on covariates and the time in the previous state. An estimation procedure for the underlying distributions and the model coefficients is suggested with the EM algorithm. The EMS algorithm (Smoothed EM algorithm) is also considered to obtain smooth estimates of the distributions. The proposed method is illustrated with data from an AIDS study and a study of patients with malignant melanoma.     

Ridge Regression for Estimation of Transition Probabilities from Aggregate Data

When data from several independent Markov chains are aggregated over each time point, least square estimation of transition probabilities faces the problem of multi-collinearity. We propose here an estimation procedure which involves use of ridge regression for the ordinary least square estimators. Performance of this estimator is then compared with that of the ordinary least squares.     

Analysis of dependently truncated data in Cox framework

Truncation is a known feature of bone marrow transplant (BMT) registry data, for which the survival time of a leukemia patient is left truncated by the waiting time to transplant. It was recently noted that a longer waiting time was linked to poorer survival. A straightforward solution is a Cox model on the survival time with the waiting time as both truncation variable and covariate. The Cox model should also include other recognized risk factors as covariates. In this article, we focus on estimating the distribution function of waiting time and the probability of selection under the aforementioned Cox model.     

Testing for state dependence in binary panel data with individual covariates by a modified quadratic exponential model

We propose a test for state dependence in binary panel data with individual covariates. For this aim, we rely on a quadratic exponential model in which the association between the response variables is accounted for in a different way with respect to more standard formulations. The level of association is measured by a single parameter that may be estimated by a Conditional Maximum Likelihood (CML) approach. Under the dynamic logit model, the conditional estimator of this parameter converges to zero when the hypothesis of absence of state dependence is true. Therefore, it is possible to implement a t-test for this hypothesis which may be very simply performed and attains the nominal significance level under several structures of the individual covariates. Through an extensive simulation study, we find that our test has good finite sample properties and it is more robust to the presence of (autocorrelated) covariates in the model specification in comparison with other existing testing procedures for state dependence. The proposed approach is illustrated by two empirical applications: the first is based on data coming from the Panel Study of Income Dynamics and concerns employment and fertility; the second is based on the Health and Retirement Study and concerns the self reported health status.     

Testing the effect of treatment on survival time with an immediate intermediate event

In this paper, we consider testing the effects of treatment on survival time when a subject experiences an immediate intermediate event (IE) prior to death or predetermined endpoint. A two-stage model incorporating both (i) the effects of the covariates on the immediate IE and (ii) survival regression with the immediate IE and other covariates is presented. We study the likelihood ratio test (LRT) for testing the treatment effect based on the proposed two stage model. We propose two procedures: an asymptotic-based procedure and a resampling-based procedure, to approximate the null distribution of the LRT. We numerically show the advantages of the two stage modeling over the existing single stage survival model with interactions between the covariates and the immediate IE. In addition, an illustrative empirical example is provided.