Analyses of the Markov modulated fluid flow with one-sided ph-type jumps using coupled queues and the completed graphs

In this paper, we analyze Markov modulated fluid flow processes with one-sided ph-type jumps using the completed graph and also through the limits of coupled queueing processes to be constructed. For the models, we derive various results on time-dependent distributions and distributions of first passage times, and present the Riccati equations that transform matrices of the first return times to 0 satisfy. The Riccati equations enable us to compute the transform matrices using Newton’s method which is known very fast and stable. Finally, we present some duality results between the model with ph-type downward jumps and the model with ph-type upward jumps. This paper contains extended results of Ahn (2009) and probabilistic interpretations given by the completed graphs.     

Granger-causality in Markov switching models

In this paper, we propose a new approach for characterizing and testing Granger-causality, which is well equipped to handle models where the change in regime evolves according to multiple Markov chains. Differently from the existing literature, we propose a method for analysing causal links that specifically takes into account Markov chains. Tests for independence are also provided. We illustrate the methodology with an empirical application, and in particular, we investigate the causality and interdependence between financial and economic cycles in USA using the bivariate Markov switching model proposed by Hamilton and Lin [13 J.D. Hamilton and J. Lin, Stock market volatility and business cycle, J. Appl. Econ. 11(5) (1996), pp. 573–593. doi: 10.1002/(SICI)1099-1255(199609)11:5<573::AID-JAE413>3.0.CO;2-T[Crossref], [Web of Science ®] , [Google Scholar]]. We find that financial variables are useful in forecasting the aggregate economic activity, and vice versa.     

Markov chain models for multivariate repeated binary data analysis

Repeated categorical outcomes frequently occur in clinical trials. Muenz and Rubinstein (1985) presented Markov chain models to analyze binary repeated data in a breast cancer study. We extend their method to the setting when more than one repeated outcome variable is of interest. In a randomized clinical trial of breast cancer, we investigate the dependency of toxicities on predictor variables and the relationship among multiple toxic effects.     

Bayesian analysis of latent Markov models with non-ignorable missing data

Latent Markov models (LMMs) are widely used in the analysis of heterogeneous longitudinal data. However, most existing LMMs are developed in fully observed data without missing entries. The main objective of this study is to develop a Bayesian approach for analyzing the LMMs with non-ignorable missing data. Bayesian methods for estimation and model comparison are discussed. The empirical performance of the proposed methodology is evaluated through simulation studies. An application to a data set derived from National Longitudinal Survey of Youth 1997 is presented.     

Markov models of mite movements

Continuous time Markov models were used to analyse data from two bioassays to investigate the influence of β-fraction, a by-product of hop processing, on the two-spotted spider mite. The models were fitted to aggregate counts of the numbers of live and dead mites on treated and untreated halves of discs cut from leaves of hop and French bean plants. Some of the rate parameters were time dependent. Although not all parameters could be estimated precisely, the analysis enabled the quantitative effects of treatment over time to be estimated with reasonable precision. The estimated treatment effects were largely insensitive to the assumed values of other parameters. The first bioassay showed a progressive initial response to increasing concentration of β-fraction, although data at the intermediate concentration appeared anomalous. The second bioassay showed similar responses on hop and French bean leaves, with a stronger repellent effect on the lower leaf surface than on the upper surface.     

Distribution of busy period in stochastic fluid models

We consider the busy period in a stochastic fluid flow model with infinite buffer where the input and output rates are controlled by a finite homogeneous Markov process. We derive an explicit expression for the distribution of the busy period and we obtain an algorithm to compute it which exhibits nice numerical properties.

     

Bayesian image analysis with Markov chain Monte Carlo and coloured continuum triangulation models

It is now possible to carry out Bayesian image segmentation from a continuum parametric model with an unknown number of regions. However, few suitable parametric models exist. We set out to model processes which have realizations that are naturally described by coloured planar triangulations. Triangulations are already used, to represent image structure in machine vision, and in finite element analysis, for domain decomposition. However, no normalizable parametric model, with realizations that are coloured triangulations, has been specified to date. We show how this must be done, and in particular we prove that a normalizable measure on the space of triangulations in the interior of a fixed simple polygon derives from a Poisson point process of vertices. We show how such models may be analysed by using Markov chain Monte Carlo methods and we present two case-studies, including convergence analysis.     

A transitional Markov switching autoregressive model

ABSTRACT

This paper is concerned with properties of a transitional Markov switching autoregressive (TMSAR) model, together with its maximum-likelihood estimation and inference. We extend existing MSAR models by allowing dependence of AR parameters on hidden states at time points prior to the current time t. A stationary solution is given and expressions for the theoretical autocovariance function are derived. Two time series are analyzed and the new model outperforms two existing MSAR models in terms of maximized log-likelihood, residual correlations, and one-step-ahead forecasting performance. The new model also gives more regime changes in agreement with real events.     

Global and local tests to assess stationarity of Markov transition models

We present global and local likelihood-based tests to evaluate stationarity in transition models. Three motivational studies are considered. A simulation study was carried out to assess the performance of the proposed tests. The results showed that they present good performance with the control of the type-I error, especially for ordinal responses, and control of the type-II error, especially for the nominal case. Asymptotically they are close to the classical test performance. They can be executed in a single framework without the need to estimate the transition probabilities, incorporating both categorical and continuous covariates, and used to identify sources of non-stationarity.     

Penalized maximum likelihood estimation for Gaussian hidden Markov models

ABSTRACT

The likelihood function of a Gaussian hidden Markov model is unbounded, which is why the maximum likelihood estimator (MLE) is not consistent. A penalized MLE is introduced along with a rigorous consistency proof.     

Robust fitting of hidden Markov regression models under a longitudinal setting

We propose a robust estimation procedure for the analysis of longitudinal data including a hidden process to account for unobserved heterogeneity between subjects in a dynamic fashion. We show how to perform estimation by an expectation–maximization-type algorithm in the hidden Markov regression literature. We show that the proposed robust approaches work comparably to the maximum-likelihood estimator when there are no outliers and the error is normal and outperform it when there are outliers or the error is heavy tailed. A real data application is used to illustrate our proposal. We also provide details on a simple criterion to choose the number of hidden states.     

Reversible jump and the label switching problem in hidden Markov models

Reversible jump Markov chain Monte Carlo (RJMCMC) algorithms can be efficiently applied in Bayesian inference for hidden Markov models (HMMs), when the number of latent regimes is unknown. As for finite mixture models, when priors are invariant to the relabelling of the regimes, HMMs are unidentifiable in data fitting, because multiple ways to label the regimes can alternate during the MCMC iterations; this is the so-called label switching problem. HMMs with an unknown number of regimes are considered here and the goal of this paper is the comparison, both applied and theoretical, of five methods used for tackling label switching within a RJMCMC algorithm; they are: post-processing, partial reordering, permutation sampling, sampling from a Markov prior and rejection sampling. The five strategies we compare have been proposed mostly in the literature of finite mixture models and only two of them, i.e. rejection sampling and partial reordering, have been presented in RJMCMC algorithms for HMMs. We consider RJMCMC algorithms in which the parameters are updated by Gibbs sampling and the dimension of the model changes in split-and-merge and birth-and-death moves. Finally, an example illustrates and compares the five different methodologies.     

The strong laws of large numbers for countable non homogeneous hidden Markov models

In this article, we are going to study the strong laws of large numbers for countable non homogeneous hidden Markov models. First, we introduce the notion of countable non homogeneous hidden Markov models. Then, we obtain some properties for those Markov models. Finally, we establish two strong laws of large numbers for countable non homogeneous hidden Markov models. As corollaries, we obtain some known results of strong laws of large numbers for finite non homogeneous Markov chains.     

A transient analysis of Markov fluid models with jumps

In this paper, we consider Markov fluid models with jumps which are useful for e.g. insurance risk modeling and the performance analysis of high-speed data networks. Recently, Ahn and Ramaswami [Ahn, S. & Ramaswami, V. (2004). Transient analysis of fluid flow models via stochastic coupling to a queue. Stochastic Models, 20 (1) 71–101] provided a transient analysis of the Markov modulated fluid flow model using stochastic coupling to a queueing model. Here we extend their results and provide a transient analysis of Markov fluid models with jumps. We also present some numerical examples.     

Multivariate Markov switching dynamic conditional correlation GARCH representations for contagion analysis

This paper provides an extension of the Dynamic Conditional Correlation model of Engle (2002) by allowing both the unconditional correlation and the parameters to be driven by an unobservable Markov chain. We provide the estimation algorithm and perform an empirical analysis of the contagion phenomenon in which our model is compared to the traditional CCC and DCC representations. We acknowledge financial support from the Italian national research project on "The Euro and European financial market volatility: contagion, interdependence and volatility transmission" financed by the Italian Ministry of University and Research. Furthermore, we thank William De Pieri for research assistance and are grateful to Loriana Pelizzon, Claudio Pizzi, Domenico Sartore and the participants at the Forecasting Financial Markets 2004 conference and at the XLII Annual Meeting of the Italian Statistical Society for helpful comments. Usual disclaimer applies. Correspondence to: Monica Bilio     

Markov random-field models for estimating local labour markets

Summary. This work is motivated by data on daily travel-to-work flows observed between pairs of elemental territorial units of an Italian region. The data were collected during the 1991 population census. The aim of the analysis is to partition the region into local labour markets. We present a new method for this which is inspired by the Bayesian texture segmentation approach. We introduce a novel Markov random-field model for the distribution of the variables that label the local labour markets for each territorial unit. Inference is performed by means of Markov chain Monte Carlo methods. The issue of model hyperparameter estimation is also addressed. We compare the results with those obtained by applying a classical method. The methodology can be applied with minor modifications to other data sets.     

Bayesian inference for generalized additive mixed models based on Markov random field priors

Most regression problems in practice require flexible semiparametric forms of the predictor for modelling the dependence of responses on covariates. Moreover, it is often necessary to add random effects accounting for overdispersion caused by unobserved heterogeneity or for correlation in longitudinal or spatial data. We present a unified approach for Bayesian inference via Markov chain Monte Carlo simulation in generalized additive and semiparametric mixed models. Different types of covariates, such as the usual covariates with fixed effects, metrical covariates with non-linear effects, unstructured random effects, trend and seasonal components in longitudinal data and spatial covariates, are all treated within the same general framework by assigning appropriate Markov random field priors with different forms and degrees of smoothness. We applied the approach in several case-studies and consulting cases, showing that the methods are also computationally feasible in problems with many covariates and large data sets. In this paper, we choose two typical applications.     

Network-based genomic discovery: application and comparison of Markov random field models

Summary.  As biological knowledge accumulates rapidly, gene networks encoding genomewide gene–gene interactions have been constructed. As an improvement over the standard mixture model that tests all the genes identically and independently distributed a priori , Wei and co-workers have proposed modelling a gene network as a discrete or Gaussian Markov random field (MRF) in a mixture model to analyse genomic data. However, how these methods compare in practical applications is not well understood and this is the aim here. We also propose two novel constraints in prior specifications for the Gaussian MRF model and a fully Bayesian approach to the discrete MRF model. We assess the accuracy of estimating the false discovery rate by posterior probabilities in the context of MRF models. Applications to a chromatin immuno-precipitation–chip data set and simulated data show that the modified Gaussian MRF models have superior performance compared with other models, and both MRF-based mixture models, with reasonable robustness to misspecified gene networks, outperform the standard mixture model.     

Spectral representation and autocovariance structure of Markov switching DSGE models

Abstract

We investigate the L²-structure of Markov switching Dynamic Stochastic General Equilibrium (MS DSGE) models and derive conditions for strict and second-order stationarity. Then we determine the autocovariance function of the process driven by a stationary MS DSGE model and give a stable VARMA representation of it. It turns out that the autocovariance structure of the process coincides with that of a standard VARMA. Finally, we propose a method to derive the spectral density in a matrix closed-form of MS DSGE models. Our results relate with the works of Francq and Zakoian, Krolzig, Zhang and Stine. Numerical and empirical illustrations complete the article.     

Using hidden Markov models with raw,triaxial wrist accelerometry data to determine sleep stages

Accelerometry is a low‐cost and noninvasive method that has been used to discriminate sleep from wake, however, its utility to detect sleep stages is unclear. We detail the development and comparison of methods which utilise raw, triaxial accelerometry data to classify varying stages of sleep, ranging from sleep/wake detection to discriminating rapid eye movement sleep, stage one sleep, stage two sleep, deep sleep and wake. First‐ and second‐order hidden Markov models (HMMs) with time‐homogeneous and time‐varying transition probability matrices, along with continuous acceleration observations in the form of a Gaussian‐observation HMM and K‐means classified acceleration in a discrete‐observation HMM were explored. In addition, generalised linear mixed models (GLMMs) with binary and multinomial responses and logit link functions were considered as was whether incorporating adjoining acceleration information into the models improved prediction. Model predictions were compared to the reference‐standard in sleep detection (polysomnography) and outcome accuracies were calculated. Consistently, HMMs yielded greater sleep stage detection than GLMMs but there was little difference between first‐ and second‐order HMMs. Varying degrees of difference were observed when comparing Gaussian‐observation HMMs to discrete‐observation HMMs, and time‐varying HMMs yielded greater discrimination than time‐homogeneous HMMs, as did models which considered adjoining acceleration information. These results suggest that wrist‐worn accelerometry data may be able to detect sleep stages but that further investigation is required to optimise classification accuracy.