Estimation and tests for a longitudinal regression model based on the Markov chain

In this paper, the dependence of transition probabilities on covariates and a test procedure for covariate dependent Markov models are examined. The nonparametric test for the role of waiting time proposed by Jones and Crowley [M. Jones, J. Crowley, Nonparametric tests of the Markov model for survival data Biometrika 79 (3) (1992) 513–522] has been extended here to transitions and reverse transitions. The limitation of the Jones and Crowley method is that it does not take account of other covariates that might have association with the probabilities of transition. A simple test procedure is proposed that can be employed for testing: (i) the significance of association between covariates and transition probabilities, and (ii) the impact of waiting time on the transition probabilities. The procedure is illustrated using panel data on hospitalization of the elderly population in the USA from the Health and Retirement Survey (HRS).     

Bayesian inference in hidden Markov models through the reversible jump Markov chain Monte Carlo method

Hidden Markov models form an extension of mixture models which provides a flexible class of models exhibiting dependence and a possibly large degree of variability. We show how reversible jump Markov chain Monte Carlo techniques can be used to estimate the parameters as well as the number of components of a hidden Markov model in a Bayesian framework. We employ a mixture of zero-mean normal distributions as our main example and apply this model to three sets of data from finance, meteorology and geomagnetism.     

The Access/Impact Problem and the Green and Gold Roads to Open Access

The research access/impact problem arises because journal articles are not accessible to all of their would-be users; hence, they are losing potential research impact. The solution is to make all articles Open Access (OA; i.e., accessible online, free for all). OA articles have significantly higher citation impact than non-OA articles. There are two roads to OA: the “golden” road (publish your article in an OA journal) and the “green” road (publish your article in a non-OA journal but also self-archive it in an OA archive). Only 5% of journals are gold, but over 90% are already green (i.e., they have given their authors the green light to self-archive); yet only about 10–20% of articles have been self-archived. To reach 100% OA, self-archiving needs to be mandated by researchers' employers and funders, as the United Kingdom and the United States have recently recommended, and universities need to implement that mandate.     

Network-based genomic discovery: application and comparison of Markov random field models

Summary.  As biological knowledge accumulates rapidly, gene networks encoding genomewide gene–gene interactions have been constructed. As an improvement over the standard mixture model that tests all the genes identically and independently distributed a priori , Wei and co-workers have proposed modelling a gene network as a discrete or Gaussian Markov random field (MRF) in a mixture model to analyse genomic data. However, how these methods compare in practical applications is not well understood and this is the aim here. We also propose two novel constraints in prior specifications for the Gaussian MRF model and a fully Bayesian approach to the discrete MRF model. We assess the accuracy of estimating the false discovery rate by posterior probabilities in the context of MRF models. Applications to a chromatin immuno-precipitation–chip data set and simulated data show that the modified Gaussian MRF models have superior performance compared with other models, and both MRF-based mixture models, with reasonable robustness to misspecified gene networks, outperform the standard mixture model.     

Bayesian regression model for recurrent event data with event-varying covariate effects and event effect

In the course of hypertension, cardiovascular disease events (e.g. stroke, heart failure) occur frequently and recurrently. The scientific interest in such study may lie in the estimation of treatment effect while accounting for the correlation among event times. The correlation among recurrent event times comes from two sources: subject-specific heterogeneity (e.g. varied lifestyles, genetic variations, and other unmeasurable effects) and event dependence (i.e. event incidences may change the risk of future recurrent events). Moreover, event incidences may change the disease progression so that there may exist event-varying covariate effects (the covariate effects may change after each event) and event effect (the effect of prior events on the future events). In this article, we propose a Bayesian regression model that not only accommodates correlation among recurrent events from both sources, but also explicitly characterizes the event-varying covariate effects and event effect. This model is especially useful in quantifying how the incidences of events change the effects of covariates and risk of future events. We compare the proposed model with several commonly used recurrent event models and apply our model to the motivating lipid-lowering trial (LLT) component of the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT) (ALLHAT-LLT).     

Applying a Markov chain for the stock pricing of a novel forecasting model

In this article, a stock-forecasting model is developed to analyze a company's stock price variation related to the Taiwanese company HTC. The main difference to previous articles is that this study uses the data of the HTC in recent ten years to build a Markov transition matrix. Instead of trying to predict the stock price variation through the traditional approach to the HTC stock problem, we integrate two types of Markov chain that are used in different ways. One is a regular Markov chain, and the other is an absorbing Markov chain. Through a regular Markov chain, we can obtain important information such as what happens in the long run or whether the distribution of the states tends to stabilize over time in an efficient way. Next, we used an artificial variable technique to create an absorbing Markov chain. Thus, we used an absorbing Markov chain to provide information about the period between the increases before arriving at the decreasing state of the HTC stock. We provide investors with information on how long the HTC stock will keep increasing before its price begins to fall, which is extremely important information to them.     

On-Line Learning for the Infinite Hidden Markov Model

We develop a sequential Monte Carlo algorithm for the infinite hidden Markov model (iHMM) that allows us to perform on-line inferences on both system states and structural (static) parameters. The algorithm described here provides a natural alternative to Markov chain Monte Carlo samplers previously developed for the iHMM, and is particularly helpful in applications where data is collected sequentially and model parameters need to be continuously updated. We illustrate our approach in the context of both a simulation study and a financial application.     

Semiparametric hidden Markov model with non-parametric regression

The hidden Markov model regression (HMMR) has been popularly used in many fields such as gene expression and activity recognition. However, the traditional HMMR requires the strong linearity assumption for the emission model. In this article, we propose a hidden Markov model with non-parametric regression (HMM-NR), where the mean and variance of emission model are unknown smooth functions. The new semiparametric model might greatly reduce the modeling bias and thus enhance the applicability of the traditional hidden Markov model regression. We propose an estimation procedure for the transition probability matrix and the non-parametric mean and variance functions by combining the ideas of the EM algorithm and the kernel regression. Simulation studies and a real data set application are used to demonstrate the effectiveness of the new estimation procedure.     

Sparse Markov Chains for Sequence Data

Finite memory sources and variable‐length Markov chains have recently gained popularity in data compression and mining, in particular, for applications in bioinformatics and language modelling. Here, we consider denser data compression and prediction with a family of sparse Bayesian predictive models for Markov chains in finite state spaces. Our approach lumps transition probabilities into classes composed of invariant probabilities, such that the resulting models need not have a hierarchical structure as in context tree‐based approaches. This can lead to a substantially higher rate of data compression, and such non‐hierarchical sparse models can be motivated for instance by data dependence structures existing in the bioinformatics context. We describe a Bayesian inference algorithm for learning sparse Markov models through clustering of transition probabilities. Experiments with DNA sequence and protein data show that our approach is competitive in both prediction and classification when compared with several alternative methods on the basis of variable memory length.     

Alternative Markov Properties for Chain Graphs

Graphical Markov models use graphs to represent possible dependences among statistical variables. Lauritzen, Wermuth, and Frydenberg (LWF) introduced a Markov property for chain graphs (CG): graphs that can be used to represent both structural and associative dependences simultaneously and that include both undirected graphs (UG) and acyclic directed graphs (ADG) as special cases. Here an alternative Markov property (AMP) for CGs is introduced and shown to be the Markov property satisfied by a block-recursive linear system with multivariate normal errors. This model can be decomposed into a collection of conditional normal models, each of which combines the features of multivariate linear regression models and covariance selection models, facilitating the estimation of its parameters. In the general case, necessary and sufficient conditions are given for the equivalence of the LWF and AMP Markov properties of a CG, for the AMP Markov equivalence of two CGs, for the AMP Markov equivalence of a CG to some ADG or decomposable UG, and for other equivalences. For CGs, in some ways the AMP property is a more direct extension of the ADG Markov property than is the LWF property.     

Statistical Inference for Partially Hidden Markov Models

ABSTRACT

In this article we introduce a new missing data model, based on a standard parametric Hidden Markov Model (HMM), for which information on the latent Markov chain is given since this one reaches a fixed state (and until it leaves this state). We study, under mild conditions, the consistency and asymptotic normality of the maximum likelihood estimator. We point out also that the underlying Markov chain does not need to be ergodic, and that identifiability of the model is not tractable in a simple way (unlike standard HMMs), but can be studied using various technical arguments.     

A latent Gaussian Markov random-field model for spatiotemporal rainfall disaggregation

Summary. Rainfall data are often collected at coarser spatial scales than required for input into hydrology and agricultural models. We therefore describe a spatiotemporal model which allows multiple imputation of rainfall at fine spatial resolutions, with a realistic dependence structure in both space and time and with the total rainfall at the coarse scale consistent with that observed. The method involves the transformation of the fine scale rainfall to a thresholded Gaussian process which we model as a Gaussian Markov random field. Gibbs sampling is then used to generate realizations of rainfall efficiently at the fine scale. Results compare favourably with previous, less elegant methods.     

Hierarchical Bayesian Approach to a Multi-Site Hidden Markov Model

Multivariate data with a sequential or temporal structure occur in various fields of study. The hidden Markov model (HMM) provides an attractive framework for modeling long-term persistence in areas of pattern recognition through the extension of independent and identically distributed mixture models. Unlike in typical mixture models, the heterogeneity of data is represented by hidden Markov states. This article extends the HMM to a multi-site or multivariate case by taking a hierarchical Bayesian approach. This extension has many advantages over a single-site HMM. For example, it can provide more information for identifying the structure of the HMM than a single-site analysis. We evaluate the proposed approach by exploiting a spatial correlation that depends on the distance between sites.     

Perfect simulation of an inventory model for perishable products

We study an inventory model for perishable products with a critical-number ordering policy under the assumption that demand for the product forms an i.i.d. sequence, so that the state of the system forms a Markov chain. Explicit calculation of the stationary distribution has proved impractical in cases where items have reasonably long lifetimes and for systems with large under-up-to levels. Using the recently developed coupling-from-the-past method, we introduce a technique to estimate the stationary distribution of the Markov chain via perfect simulation. The Markov chain that results from the use of a critical-number policy is particularly amenable to these simulation techniques, despite not being ordered in its initial state, since the recursive equations satisfied by the Markov chain enable us to identify specific demand patterns where the backward coupling occurs.     

Estimation of Dynamic Discrete Choice Models Using Artificial Neural Network Approximations

I propose a method for inference in dynamic discrete choice models (DDCM) that utilizes Markov chain Monte Carlo (MCMC) and artificial neural networks (ANNs). MCMC is intended to handle high-dimensional integration in the likelihood function of richly specified DDCMs. ANNs approximate the dynamic-program (DP) solution as a function of the parameters and state variables prior to estimation to avoid having to solve the DP on each iteration. Potential applications of the proposed methodology include inference in DDCMs with random coefficients, serially correlated unobservables, and dependence across individual observations. The article discusses MCMC estimation of DDCMs, provides relevant background on ANNs, and derives a theoretical justification for the method. Experiments suggest this to be a promising approach.     

Mixture Multi-state Markov Regression Model

Although heterogeneity across individuals may be reduced when a two-state process is extended into a multi-state process, the discrepancy between the observed and the predicted for some states may still exist owing to two possibilities, unobserved mixture distribution in the initial state and the effect of measured covariates on subsequent multi-state disease progression. In the present study, we developed a mixture Markov exponential regression model to take account of the above-mentioned heterogeneity across individuals (subject-to-subject variability) with a systematic model selection based on the likelihood ratio test. The model was successfully demonstrated by an empirical example on surveillance of patients with small hepatocellular carcinoma treated by non-surgical methods. The estimated results suggested that the model with the incorporation of unobserved mixture distribution behaves better than the one without. Complete and partial effects regarding risk factors on different subsequent multi-state transitions were identified using a homogeneous Markov model. The combination of both initial mixture distribution and homogeneous Markov exponential regression model makes a significant contribution to reducing heterogeneity across individuals and over time for disease progression.     

Reversible jump and the label switching problem in hidden Markov models

Reversible jump Markov chain Monte Carlo (RJMCMC) algorithms can be efficiently applied in Bayesian inference for hidden Markov models (HMMs), when the number of latent regimes is unknown. As for finite mixture models, when priors are invariant to the relabelling of the regimes, HMMs are unidentifiable in data fitting, because multiple ways to label the regimes can alternate during the MCMC iterations; this is the so-called label switching problem. HMMs with an unknown number of regimes are considered here and the goal of this paper is the comparison, both applied and theoretical, of five methods used for tackling label switching within a RJMCMC algorithm; they are: post-processing, partial reordering, permutation sampling, sampling from a Markov prior and rejection sampling. The five strategies we compare have been proposed mostly in the literature of finite mixture models and only two of them, i.e. rejection sampling and partial reordering, have been presented in RJMCMC algorithms for HMMs. We consider RJMCMC algorithms in which the parameters are updated by Gibbs sampling and the dimension of the model changes in split-and-merge and birth-and-death moves. Finally, an example illustrates and compares the five different methodologies.     

Markov Chain Markov Field dynamics: Models and statistics

This study deals with time dynamics of Markov fields defined on a finite set of sites with state space <$>E<$>, focussing on Markov Chain Markov Field (MCMF) evolution. Such a model is characterized by two families of potentials: the instantaneous interaction potentials, and the time delay potentials. Four models are specified: auto-exponential dynamics (<$>E = {\of R}^+<$>), auto-normal dynamics (<$>E = {\of R}<$>), auto-Poissonian dynamics (<$>E = {\of N}<$>) and auto-logistic dynamics ( E qualitative and finite). Sufficient conditions ensuring ergodicity and strong law of large numbers are given by using a Lyapunov criterion of stability, and the conditional pseudo-likelihood statistics are summarized. We discuss the identification procedure of the two Markovian graphs and look for validation tests using martingale central limit theorems. An application to meteorological data illustrates such a modelling.     

Stationary distributions for a class of Markov-modulated tandem fluid queues

ABSTRACT

We consider a model consisting of two fluid queues driven by the same background continuous-time Markov chain, such that the rates of change of the fluid in the second queue depend on whether the first queue is empty or not: when the first queue is nonempty, the content of the second queue increases, and when the first queue is empty, the content of the second queue decreases.

We analyze the stationary distribution of this tandem model using operator-analytic methods. The various densities (or Laplace–Stieltjes transforms thereof) and probability masses involved in this stationary distribution are expressed in terms of the stationary distribution of some embedded process. To find the latter from the (known) transition kernel, we propose a numerical procedure based on discretization and truncation. For some examples we show the method works well, although its performance is clearly affected by the quality of these approximations, both in terms of accuracy and run time.