Nonparametric Bayesian analysis for multi-site hidden Markov model

The hidden Markov model (HMM) provides an attractive framework for modeling long-term persistence in a variety of applications including pattern recognition. Unlike typical mixture models, hidden Markov states can represent the heterogeneity in data and it can be extended to a multivariate case using a hierarchical Bayesian approach. This article provides a nonparametric Bayesian modeling approach to the multi-site HMM by considering stick-breaking priors for each row of an infinite state transition matrix. This extension has many advantages over a parametric HMM. For example, it can provide more flexible information for identifying the structure of the HMM than parametric HMM analysis, such as the number of states in HMM. We exploit a simulation example and a real dataset to evaluate the proposed approach.     

Testing for two states in a hidden Markov model

The authors consider hidden Markov models (HMMs) whose latent process has m ≥ 2 states and whose state‐dependent distributions arise from a general one‐parameter family. They propose a test of the hypothesis m = 2. Their procedure is an extension to HMMs of the modified likelihood ratio statistic proposed by Chen, Chen & Kalbfleisch (2004) for testing two states in a finite mixture. The authors determine the asymptotic distribution of their test under the hypothesis m = 2 and investigate its finite‐sample properties in a simulation study. Their test is based on inference for the marginal mixture distribution of the HMM. In order to illustrate the additional difficulties due to the dependence structure of the HMM, they show how to test general regular hypotheses on the marginal mixture of HMMs via a quasi‐modified likelihood ratio. They also discuss two applications.     

Hierarchical Bayesian Approach to a Multi-Site Hidden Markov Model

Multivariate data with a sequential or temporal structure occur in various fields of study. The hidden Markov model (HMM) provides an attractive framework for modeling long-term persistence in areas of pattern recognition through the extension of independent and identically distributed mixture models. Unlike in typical mixture models, the heterogeneity of data is represented by hidden Markov states. This article extends the HMM to a multi-site or multivariate case by taking a hierarchical Bayesian approach. This extension has many advantages over a single-site HMM. For example, it can provide more information for identifying the structure of the HMM than a single-site analysis. We evaluate the proposed approach by exploiting a spatial correlation that depends on the distance between sites.     

Reversible jump and the label switching problem in hidden Markov models

Reversible jump Markov chain Monte Carlo (RJMCMC) algorithms can be efficiently applied in Bayesian inference for hidden Markov models (HMMs), when the number of latent regimes is unknown. As for finite mixture models, when priors are invariant to the relabelling of the regimes, HMMs are unidentifiable in data fitting, because multiple ways to label the regimes can alternate during the MCMC iterations; this is the so-called label switching problem. HMMs with an unknown number of regimes are considered here and the goal of this paper is the comparison, both applied and theoretical, of five methods used for tackling label switching within a RJMCMC algorithm; they are: post-processing, partial reordering, permutation sampling, sampling from a Markov prior and rejection sampling. The five strategies we compare have been proposed mostly in the literature of finite mixture models and only two of them, i.e. rejection sampling and partial reordering, have been presented in RJMCMC algorithms for HMMs. We consider RJMCMC algorithms in which the parameters are updated by Gibbs sampling and the dimension of the model changes in split-and-merge and birth-and-death moves. Finally, an example illustrates and compares the five different methodologies.     

Using hidden Markov models with raw,triaxial wrist accelerometry data to determine sleep stages

Accelerometry is a low‐cost and noninvasive method that has been used to discriminate sleep from wake, however, its utility to detect sleep stages is unclear. We detail the development and comparison of methods which utilise raw, triaxial accelerometry data to classify varying stages of sleep, ranging from sleep/wake detection to discriminating rapid eye movement sleep, stage one sleep, stage two sleep, deep sleep and wake. First‐ and second‐order hidden Markov models (HMMs) with time‐homogeneous and time‐varying transition probability matrices, along with continuous acceleration observations in the form of a Gaussian‐observation HMM and K‐means classified acceleration in a discrete‐observation HMM were explored. In addition, generalised linear mixed models (GLMMs) with binary and multinomial responses and logit link functions were considered as was whether incorporating adjoining acceleration information into the models improved prediction. Model predictions were compared to the reference‐standard in sleep detection (polysomnography) and outcome accuracies were calculated. Consistently, HMMs yielded greater sleep stage detection than GLMMs but there was little difference between first‐ and second‐order HMMs. Varying degrees of difference were observed when comparing Gaussian‐observation HMMs to discrete‐observation HMMs, and time‐varying HMMs yielded greater discrimination than time‐homogeneous HMMs, as did models which considered adjoining acceleration information. These results suggest that wrist‐worn accelerometry data may be able to detect sleep stages but that further investigation is required to optimise classification accuracy.     

Statistical Inference for Partially Hidden Markov Models

ABSTRACT

In this article we introduce a new missing data model, based on a standard parametric Hidden Markov Model (HMM), for which information on the latent Markov chain is given since this one reaches a fixed state (and until it leaves this state). We study, under mild conditions, the consistency and asymptotic normality of the maximum likelihood estimator. We point out also that the underlying Markov chain does not need to be ergodic, and that identifiability of the model is not tractable in a simple way (unlike standard HMMs), but can be studied using various technical arguments.     

Robust fitting of hidden Markov regression models under a longitudinal setting

We propose a robust estimation procedure for the analysis of longitudinal data including a hidden process to account for unobserved heterogeneity between subjects in a dynamic fashion. We show how to perform estimation by an expectation–maximization-type algorithm in the hidden Markov regression literature. We show that the proposed robust approaches work comparably to the maximum-likelihood estimator when there are no outliers and the error is normal and outperform it when there are outliers or the error is heavy tailed. A real data application is used to illustrate our proposal. We also provide details on a simple criterion to choose the number of hidden states.     

Likelihood Ratio Testing for Hidden Markov Models Under Non-standard Conditions

Abstract.  In practical applications, when testing parametric restrictions for hidden Markov models (HMMs), one frequently encounters non-standard situations such as testing for zero entries in the transition matrix, one-sided tests for the parameters of the transition matrix or for the components of the stationary distribution of the underlying Markov chain, or testing boundary restrictions on the parameters of the state-dependent distributions. In this paper, we briefly discuss how the relevant asymptotic distribution theory for the likelihood ratio test (LRT) when the true parameter is on the boundary extends from the independent and identically distributed situation to HMMs. Then we concentrate on discussing a number of relevant examples. The finite-sample performance of the LRT in such situations is investigated in a simulation study. An application to series of epileptic seizure counts concludes the paper.     

Estimating the order of a hidden markov model

While the estimation of the parameters of a hidden Markov model has been studied extensively, the consistent estimation of the number of hidden states is still an unsolved problem. The AIC and BIC methods are used most commonly, but their use in this context has not been justified theoretically. The author shows that for many common models, the penalized minimum‐distance method yields a consistent estimate of the number of hidden states in a stationary hidden Markov model. In addition to addressing the identifiability issues, she applies her method to a multiple sclerosis data set and assesses its performance via simulation.     

VARIATIONAL BAYESIAN ANALYSIS FOR HIDDEN MARKOV MODELS

The variational approach to Bayesian inference enables simultaneous estimation of model parameters and model complexity. An interesting feature of this approach is that it also leads to an automatic choice of model complexity. Empirical results from the analysis of hidden Markov models with Gaussian observation densities illustrate this. If the variational algorithm is initialized with a large number of hidden states, redundant states are eliminated as the method converges to a solution, thereby leading to a selection of the number of hidden states. In addition, through the use of a variational approximation, the deviance information criterion for Bayesian model selection can be extended to the hidden Markov model framework. Calculation of the deviance information criterion provides a further tool for model selection, which can be used in conjunction with the variational approach.     

ESTIMATING COMPONENTS IN FINITE MIXTURES AND HIDDEN MARKOV MODELS

When the unobservable Markov chain in a hidden Markov model is stationary the marginal distribution of the observations is a finite mixture with the number of terms equal to the number of the states of the Markov chain. This suggests the number of states of the unobservable Markov chain can be estimated by determining the number of mixture components in the marginal distribution. This paper presents new methods for estimating the number of states in a hidden Markov model, and coincidentally the unknown number of components in a finite mixture, based on penalized quasi‐likelihood and generalized quasi‐likelihood ratio methods constructed from the marginal distribution. The procedures advocated are simple to calculate, and results obtained in empirical applications indicate that they are as effective as current available methods based on the full likelihood. Under fairly general regularity conditions, the methods proposed generate strongly consistent estimates of the unknown number of states or components.     

A grouped beta process model for multivariate resting‐state EEG microstate analysis on twins

EEG microstate analysis investigates the collection of distinct temporal blocks that characterize the electrical activity of the brain. Brain activity within each microstate is stable, but activity switches rapidly between different microstates in a nonrandom way. We propose a Bayesian nonparametric model that concurrently estimates the number of microstates and their underlying behaviour. We use a Markov switching vector autoregressive (VAR) framework, where a hidden Markov model (HMM) controls the nonrandom state switching dynamics of the EEG activity and a VAR model defines the behaviour of all time points within a given state. We analyze the resting‐state EEG data from twin pairs collected through the Minnesota Twin Family Study, consisting of 70 epochs per participant, where each epoch corresponds to 2 s of EEG data. We fit our model at the twin pair level, sharing information within epochs from the same participant and within epochs from the same twin pair. We capture within twin‐pair similarity, using an Indian buffet process, to consider an infinite library of microstates, allowing each participant to select a finite number of states from this library. The state spaces of highly similar twins may completely overlap while dissimilar twins could select distinct state spaces. In this way, our Bayesian nonparametric model defines a sparse set of states that describe the EEG data. All epochs from a single participant use the same set of states and are assumed to adhere to the same state switching dynamics in the HMM model, enforcing within‐participant similarity.     

Markov switching model of nonlinear autoregressive with skew-symmetric innovations

We consider data generating structures which can be represented as a Markov switching of nonlinear autoregressive model with considering skew-symmetric innovations such that switching between the states is controlled by a hidden Markov chain. We propose semi-parametric estimators for the nonlinear functions of the proposed model based on a maximum likelihood (ML) approach and study sufficient conditions for geometric ergodicity of the process. Also, an Expectation-Maximization type optimization for obtaining the ML estimators are presented. A simulation study and a real world application are also performed to illustrate and evaluate the proposed methodology.     

A hierarchical model for space–time surveillance data on meningococcal disease incidence

Summary. We describe a model-based approach to analyse space–time surveillance data on meningococcal disease. Such data typically comprise a number of time series of disease counts, each representing a specific geographical area. We propose a hierarchical formulation, where latent parameters capture temporal, seasonal and spatial trends in disease incidence. We then add—for each area—a hidden Markov model to describe potential additional (autoregressive) effects of the number of cases at the previous time point. Different specifications for the functional form of this autoregressive term are compared which involve the number of cases in the same or in neighbouring areas. The two states of the Markov chain can be interpreted as representing an 'endemic' and a 'hyperendemic' state. The methodology is applied to a data set of monthly counts of the incidence of meningococcal disease in the 94 départements of France from 1985 to 1997. Inference is carried out by using Markov chain Monte Carlo simulation techniques in a fully Bayesian framework. We emphasize that a central feature of our model is the possibility of calculating—for each region and each time point—the posterior probability of being in a hyperendemic state, adjusted for global spatial and temporal trends, which we believe is of particular public health interest.     

On-Line Learning for the Infinite Hidden Markov Model

We develop a sequential Monte Carlo algorithm for the infinite hidden Markov model (iHMM) that allows us to perform on-line inferences on both system states and structural (static) parameters. The algorithm described here provides a natural alternative to Markov chain Monte Carlo samplers previously developed for the iHMM, and is particularly helpful in applications where data is collected sequentially and model parameters need to be continuously updated. We illustrate our approach in the context of both a simulation study and a financial application.     

Hidden Markov Models with mixtures as emission distributions

In unsupervised classification, Hidden Markov Models (HMM) are used to account for a neighborhood structure between observations. The emission distributions are often supposed to belong to some parametric family. In this paper, a semiparametric model where the emission distributions are a mixture of parametric distributions is proposed to get a higher flexibility. We show that the standard EM algorithm can be adapted to infer the model parameters. For the initialization step, starting from a large number of components, a hierarchical method to combine them into the hidden states is proposed. Three likelihood-based criteria to select the components to be combined are discussed. To estimate the number of hidden states, BIC-like criteria are derived. A simulation study is carried out both to determine the best combination between the combining criteria and the model selection criteria and to evaluate the accuracy of classification. The proposed method is also illustrated using a biological dataset from the model plant Arabidopsis thaliana. A R package HMMmix is freely available on the CRAN.     

A continuous-time HMM approach to modeling the magnitude-frequency distribution of earthquakes

The magnitude-frequency distribution (MFD) of earthquake is a fundamental statistic in seismology. The so-called b-value in the MFD is of particular interest in geophysics. A continuous time hidden Markov model (HMM) is proposed for characterizing the variability of b-values. The HMM-based approach to modeling the MFD has some appealing properties over the widely used sliding-window approach. Often, large variability appears in the estimation of b-value due to window size tuning, which may cause difficulties in interpretation of b-value heterogeneities. Continuous-time hidden Markov models (CT-HMMs) are widely applied in various fields. It bears some advantages over its discrete time counterpart in that it can characterize heterogeneities appearing in time series in a finer time scale, particularly for highly irregularly-spaced time series, such as earthquake occurrences. We demonstrate an expectation–maximization algorithm for the estimation of general exponential family CT-HMM. In parallel with discrete-time hidden Markov models, we develop a continuous time version of Viterbi algorithm to retrieve the overall optimal path of the latent Markov chain. The methods are applied to New Zealand deep earthquakes. Before the analysis, we first assess the completeness of catalogue events to assure the analysis is not biased by missing data. The estimation of b-value is stable over the selection of magnitude thresholds, which is ideal for the interpretation of b-value variability.     

Bayesian inference in hidden Markov models through the reversible jump Markov chain Monte Carlo method

Hidden Markov models form an extension of mixture models which provides a flexible class of models exhibiting dependence and a possibly large degree of variability. We show how reversible jump Markov chain Monte Carlo techniques can be used to estimate the parameters as well as the number of components of a hidden Markov model in a Bayesian framework. We employ a mixture of zero-mean normal distributions as our main example and apply this model to three sets of data from finance, meteorology and geomagnetism.     

Stability of a characterization by the identical distribution of linear forms

Hidden Markov models (HMMs) have during the last decade become a widely spread tool for modelling sequences of dependent random variables. Inference for HMMs has been considered by several authors, but so far no work has been done on estimating their order. In this paper we propose a penalized likelihood estimator for this purpose. This estimator is based on the m-dimensional distribution of HMM, and it is shown that in the limit it does not underestimate the order.