MULTILEVEL DISCRETE‐TIME EVENT HISTORY MODELS WITH APPLICATIONS TO THE ANALYSIS OF RECURRENT EMPLOYMENT TRANSITIONS

Data on the timing of events such as births, residential moves and changes in employment status are collected in many longitudinal surveys. These data often have a highly complex structure, with events of several types occurring repeatedly over time to an individual and interdependences between different event processes (e.g. births and employment transitions). The aim of this paper is to review a general class of multilevel discrete‐time event history models for handling recurrent events and transitions between multiple states. It is also shown how standard methods can be extended to allow for time‐varying covariates that are outcomes of an event process that is jointly determined with the process of interest. The considerable potential of these methods for studying transitions through the life course is illustrated in analyses of the effect of the presence and age of children on women's employment transitions, using data from the British Household Panel Survey.     

Earthquakes occurrences estimation through a parametric semi-Markov approach

The estimation of earthquakes’ occurrences prediction in seismic areas is a challenging problem in seismology and earthquake engineering. Indeed, the prevention and the quantification of possible damage provoked by destructive earthquakes are directly linked to this kind of prevision. In our paper, we adopt a parametric semi-Markov approach. This model assumes that a sequence of earthquakes is seen as a Markov process and besides it permits to take into consideration the more realistic assumption of events’ dependence in space and time. The elapsed time between two consecutive events is modeled as a general Weibull distribution. We determine then the transition probabilities and the so-called crossing states probabilities. We conclude then with a Monte Carlo simulation and the model is validated through a large database containing real data.     

BAYESIAN HIDDEN MARKOV MODELS FOR LONGITUDINAL COUNTS

This paper describes a Bayesian approach to modelling carcinogenity in animal studies where the data consist of counts of the number of tumours present over time. It compares two autoregressive hidden Markov models. One of them models the transitions between three latent states: an inactive transient state, a multiplying state for increasing counts and a reducing state for decreasing counts. The second model introduces a fourth tied state to describe non‐zero observations that are neither increasing nor decreasing. Both these models can model the length of stay upon entry of a state. A discrete constant hazards waiting time distribution is used to model the time to onset of tumour growth. Our models describe between‐animal‐variability by a single hierarchy of random effects and the within‐animal variation by first‐order serial dependence. They can be extended to higher‐order serial dependence and multi‐level hierarchies. Analysis of data from animal experiments comparing the influence of two genes leads to conclusions that differ from those of Dunson (2000). The observed data likelihood defines an information criterion to assess the predictive properties of the three‐ and four‐state models. The deviance information criterion is appropriately defined for discrete parameters.     

Centered parameterizations and dependence limitations in Markov random field models

Markov random field models incorporate terms representing local statistical dependence among variables in a discrete-index random field. Traditional parameterizations for models based on one-parameter exponential family conditional distributions contain components that would appear to reflect large-scale and small-scale model behaviors, and it is natural to attempt to match these structures with large-scale and small-scale patterns in a set of data. Traditional manners of parameterizing Markov random field models do not allow such correspondence, however. We propose an alternative centered parameterization that, while not leading to different models, allows a correspondence between model structures and data structures to be successfully accomplished. The ability to make these connections is important when incorporating covariate information into a model or if a sequence of models is fit over time to investigate and interpret possible changes in data structure. We demonstrate the improved interpretation that results from use of centered parameterizations. Centered parameterizations also lend themselves to computation of an interpretable decomposition of mean squared error, and this is demonstrated both analytically and through a simulated example. A breakdown in model behavior occurs even with centered parameterizations if dependence parameters in Markov random field models are allowed to become too large. This phenomenon is discussed and illustrated using an auto-logistic model.     

Structure Learning of Contextual Markov Networks using Marginal Pseudo‐likelihood

Markov networks are popular models for discrete multivariate systems where the dependence structure of the variables is specified by an undirected graph. To allow for more expressive dependence structures, several generalizations of Markov networks have been proposed. Here, we consider the class of contextual Markov networks which takes into account possible context‐specific independences among pairs of variables. Structure learning of contextual Markov networks is very challenging due to the extremely large number of possible structures. One of the main challenges has been to design a score, by which a structure can be assessed in terms of model fit related to complexity, without assuming chordality. Here, we introduce the marginal pseudo‐likelihood as an analytically tractable criterion for general contextual Markov networks. Our criterion is shown to yield a consistent structure estimator. Experiments demonstrate the favourable properties of our method in terms of predictive accuracy of the inferred models.     

A transitional Markov switching autoregressive model

ABSTRACT

This paper is concerned with properties of a transitional Markov switching autoregressive (TMSAR) model, together with its maximum-likelihood estimation and inference. We extend existing MSAR models by allowing dependence of AR parameters on hidden states at time points prior to the current time t. A stationary solution is given and expressions for the theoretical autocovariance function are derived. Two time series are analyzed and the new model outperforms two existing MSAR models in terms of maximized log-likelihood, residual correlations, and one-step-ahead forecasting performance. The new model also gives more regime changes in agreement with real events.     

The double chain markov model

In the class of discrete time Markovian processes, two models are widely used, the Markov chain and the hidden Markov model. A major difference between these two models lies in the relation between successive outputs of the observed variable. In a visible Markov chain, these are directly correlated while in hidden models they are not. However, in some situations it is possible to observe both a hidden Markov chain and a direct relation between successive observed outputs. Unfortunately, the use of either a visible or a hidden model implies the suppression of one of these hypothesis. This paper prsents a Markovian model under random environment called the Double Chain Markov Model which takes into account the maijn features of both visible and hidden models. Its main purpose is the modeling of non-homogeneous time-series. It is very flexible and can be estimated with traditional methods. The model is applied on a sequence of wind speeds and it appears to model data more successfully than both the usual Markov chains and hidden Markov models.     

Multivariate Poisson hidden Markov models with a case study of modelling seismicity

While the literature on multivariate models for continuous data flourishes, there is a lack of models for multivariate counts. We aim to contribute to this framework by extending the well known class of univariate hidden Markov models to the multidimensional case, by introducing multivariate Poisson hidden Markov models. Each state of the extended model is associated with a different multivariate discrete distribution. We consider different distributions with Poisson marginals, starting from the multivariate Poisson distribution and then extending to copula based distributions to allow flexible dependence structures. An EM type algorithm is developed for maximum likelihood estimation. A real data application is presented to illustrate the usefulness of the proposed models. In particular, we apply the models to the occurrence of strong earthquakes (surface wave magnitude ≥5), in three seismogenic subregions in the broad region of the North Aegean Sea for the time period from 1 January 1981 to 31 December 2008. Earthquakes occurring in one subregion may trigger events in adjacent ones and hence the observed time series of events are cross‐correlated. It is evident from the results that the three subregions interact with each other at times differing by up to a few months. This migration of seismic activity is captured by the model as a transition to a state of higher seismicity.     

Multistate recapture models: Modelling incomplete individual histories

Multistate capture-recapture models are a natural generalization of the usual one-site recapture models. Similarly, individuals are sampled on discrete occasions, at which they may be captured or not. However, contrary to the one-site case, the individuals can move within a finite set of states between occasions. The growing interest in spatial aspects of population dynamics presently contributes to making multistate models a very promising tool for population biology. We review first the interest and the potential of multistate models, in particular when they are used with individual states as well as geographical sites. Multistate models indeed constitute canonical capture-recapture models for individual categorical covariates changing over time, and can be linked to longitudinal studies with missing data and models such as hidden Markov chains. Multistate models also provide a promising tool for handling heterogeneity of capture, provided states related to capturability can be defined and used. Such an approach could be relevant for population size estimation in closed populations. Multistate models also constitute a natural framework for mixtures of information in individual history data. Presently, most models can be fit using program MARK. As an example, we present a canonical model for multisite accession to reproduction, which fully generalizes a classical one-site model. In the generalization proposed, one can estimate simultaneously age-dependent rates of accession to reproduction, natal and breeding dispersal. Finally, we discuss further generalizations - such as a multistate generalization of growth rate models and models for data where the state in which an individual is detected is known with uncertainty - and prospects for software development.     

Multistate recapture models: modelling incomplete individual histories

Multistate capture-recapture models are a natural generalization of the usual one-site recapture models. Similarly, individuals are sampled on discrete occasions, at which they may be captured or not. However, contrary to the one-site case, the individuals can move within a finite set of states between occasions. The growing interest in spatial aspects of population dynamics presently contributes to making multistate models a very promising tool for population biology. We review first the interest and the potential of multistate models, in particular when they are used with individual states as well as geographical sites. Multistate models indeed constitute canonical capture-recapture models for individual categorical covariates changing over time, and can be linked to longitudinal studies with missing data and models such as hidden Markov chains. Multistate models also provide a promising tool for handling heterogeneity of capture, provided states related to capturability can be defined and used. Such an approach could be relevant for population size estimation in closed populations. Multistate models also constitute a natural framework for mixtures of information in individual history data. Presently, most models can be fit using program MARK. As an example, we present a canonical model for multisite accession to reproduction, which fully generalizes a classical one-site model. In the generalization proposed, one can estimate simultaneously age-dependent rates of accession to reproduction, natal and breeding dispersal. Finally, we discuss further generalizations - such as a multistate generalization of growth rate models and models for data where the state in which an individual is detected is known with uncertainty - and prospects for software development.     

Predictive event modelling in multicenter clinical trials with waiting time to response

A new analytic statistical technique for predictive event modeling in ongoing multicenter clinical trials with waiting time to response is developed. It allows for the predictive mean and predictive bounds for the number of events to be constructed over time, accounting for the newly recruited patients and patients already at risk in the trial, and for different recruitment scenarios. For modeling patient recruitment, an advanced Poisson-gamma model is used, which accounts for the variation in recruitment over time, the variation in recruitment rates between different centers and the opening or closing of some centers in the future. A few models for event appearance allowing for 'recurrence', 'death' and 'lost-to-follow-up' events and using finite Markov chains in continuous time are considered. To predict the number of future events over time for an ongoing trial at some interim time, the parameters of the recruitment and event models are estimated using current data and then the predictive recruitment rates in each center are adjusted using individual data and Bayesian re-estimation. For a typical scenario (continue to recruit during some time interval, then stop recruitment and wait until a particular number of events happens), the closed-form expressions for the predictive mean and predictive bounds of the number of events at any future time point are derived under the assumptions of Markovian behavior of the event progression. The technique is efficiently applied to modeling different scenarios for some ongoing oncology trials. Case studies are considered.     

Expanded renal transplantation: a competing risk model approach

Multi-state models (MSMs) are useful to analyze survival data when, besides the event of main interest, one or more intermediate states of the individual are identified. These models take the several existing states and the possible transitions among them into account. At the same time, covariate effects on each transition intensity may be investigated separately and, therefore, MSMs are more flexible than the standard Cox proportional hazards model. In this work, we use MSMs to investigate the impact of the quality of a transplanted kidney for a group of patients at the Hospital Universitario Central de Asturias. Specifically, we use an illness-death model to study the evolution of patients with kidney disease who received a renal transplant after a dialysis period. The intermediate state is defined as the failure of the received organ, while the terminating state is the death of the patient. In order to increase the potential number of organs available for transplant, the standards of quality for the transplanted kidneys were relaxed (the new criteria are labeled expanded criteria), and these ‘expanded kidneys’ were transplanted in appropriate candidates (older patients, with higher prevalence of diabetes mellitus). Results suggest that the expanded kidneys have a minor effect on survival, while both the kidney mortality and the risk of death increase with the patient's age and the serum creatinine and serum hemoglobin levels.     

COMPOSITIONAL TIME SERIES ANALYSIS OF MORTALITY PROPORTIONS

Compositional time series are multivariate time series which at each time point are proportions that sum to a constant. Accurate inference for such series which occur in several disciplines such as geology, economics and ecology is important in practice. Usual multivariate statistical procedures ignore the inherent constrained nature of these observations as parts of a whole and may lead to inaccurate estimation and prediction. In this article, a regression model with vector autoregressive moving average (VARMA) errors is fit to the compositional time series after an additive log ratio (ALR) transformation. Inference is carried out in a hierarchical Bayesian framework using Markov chain Monte Carlo techniques. The approach is illustrated on compositional time series of mortality events in Los Angeles in order to investigate dependence of different categories of mortality on air quality.     

A Quantile Survival Model for Censored Data

In this paper we propose a quantile survival model to analyze censored data. This approach provides a very effective way to construct a proper model for the survival time conditional on some covariates. Once a quantile survival model for the censored data is established, the survival density, survival or hazard functions of the survival time can be obtained easily. For illustration purposes, we focus on a model that is based on the generalized lambda distribution (GLD). The GLD and many other quantile function models are defined only through their quantile functions, no closed‐form expressions are available for other equivalent functions. We also develop a Bayesian Markov Chain Monte Carlo (MCMC) method for parameter estimation. Extensive simulation studies have been conducted. Both simulation study and application results show that the proposed quantile survival models can be very useful in practice.     

Costing Mixed Coxian Phase-type Systems with Poisson Arrivals

Previously, we developed a modeling framework which classifies individuals with respect to their length of stay (LOS) in the transient states of a continuous-time Markov model with a single absorbing state; phase-type models are used for each class of the Markov model. We here add costs and obtain results for moments of total costs in (0, t], for an individual, a cohort arriving at time zero and when arrivals are Poisson. Based on stroke patient data from the Belfast City Hospital we use the overall modelling framework to obtain results for total cost in a given time interval.     

Nonparametric Bayesian analysis for multi-site hidden Markov model

The hidden Markov model (HMM) provides an attractive framework for modeling long-term persistence in a variety of applications including pattern recognition. Unlike typical mixture models, hidden Markov states can represent the heterogeneity in data and it can be extended to a multivariate case using a hierarchical Bayesian approach. This article provides a nonparametric Bayesian modeling approach to the multi-site HMM by considering stick-breaking priors for each row of an infinite state transition matrix. This extension has many advantages over a parametric HMM. For example, it can provide more flexible information for identifying the structure of the HMM than parametric HMM analysis, such as the number of states in HMM. We exploit a simulation example and a real dataset to evaluate the proposed approach.     

Analysis of band‐recovery data in a multistate capture‐recapture framework

Dead recoveries of marked animals are commonly used to estimate survival probabilities. Band‐recovery models can be parameterized either by r (the probability of recovering a band conditional on death of the animal) or by f (the probability that an animal will be killed, retrieved, and have its band reported). The T parametrization can be implemented in a capture‐recapture framework with two states (alive and newly dead), mortality being the transition probability between the two states. The authors show here that the f parametrization can also be implemented in a multistate framework by imposing simple constraints on some parameters. They illustrate it using data on the mallard and the snow goose. However, they mention that because it does not entirely separate the individual survival and encounter processes, the f parametrization must be used with care on reduced models, or in the presence of estimates at the boundary of the parameter space. As they show, a multistate framework allows the use of powerful software for model fitting or testing the goodness‐of‐fit of models; it also affords the implementation of complex models such as those based on mixture of information or uncertain states     

A Flexible Approach to Time-varying Coefficients in the Cox Regression Setting

Research on methods for studying time-to-event data (survival analysis) has been extensive in recent years. The basic model in use today represents the hazard function for an individual through a proportional hazards model (Cox, 1972). Typically, it is assumed that a covariate's effect on the hazard function is constant throughout the course of the study. In this paper we propose a method to allow for possible deviations from the standard Cox model, by allowing the effect of a covariate to vary over time. This method is based on a dynamic linear model. We present our method in terms of a Bayesian hierarchical model. We fit the model to the data using Markov chain Monte Carlo methods. Finally, we illustrate the approach with several examples. This revised version was published online in July 2006 with corrections to the Cover Date.     

A Markov chain model used in analyzing disease history applied to a stroke study

In clinical research, study subjects may experience multiple events that are observed and recorded periodically. To analyze transition patterns of disease processes, it is desirable to use those multiple events over time in the analysis. This study proposes a multi-state Markov model with piecewise transition probability, which is able to accommodate periodically observed clinical data without a time homogeneity assumption. Models with ordinal outcomes that incorporate covariates are also discussed. The proposed models are illustrated by an analysis of the severity of morbidity in a monthly follow-up study for patients with spontaneous intracerebral hemorrhage.     

Evaluating the Effectiveness of State-Switching Time Series Models for U.S. Real Output

Two types of state-switching models for U.S. real output have been proposed: models that switch randomly between states and models that switch states deterministically, as in the threshold autoregressive model of Potter. These models have been justified primarily on how well they fit the sample data, yielding statistically significant estimates of the model coefficients. Here we propose a new approach to the evaluation of an estimated nonlinear time series model that provides a complement to existing methods based on in-sample fit or on out-of-sample forecasting. In this new approach, a battery of distinct nonlinearity tests is applied to the sample data, resulting in a set of p-values for rejecting the null hypothesis of a linear generating mechanism. This set of p-values is taken to be a “stylized fact” characterizing the nonlinear serial dependence in the generating mechanism of the time series. The effectiveness of an estimated nonlinear model for this time series is then evaluated in terms of the congruence between this stylized fact and a set of nonlinearity test results obtained from data simulated using the estimated model. In particular, we derive a portmanteau statistic based on this set of nonlinearity test p-values that allows us to test the proposition that a given model adequately captures the nonlinear serial dependence in the sample data. We apply the method to several estimated state-switching models of U.S. real output.