Handling the Label Switching Problem in Latent Class Models Via the ECR Algorithm

Latent class models (LCMs) are specific cases of mixture models. Under a Bayesian setup, the symmetric posterior distribution of these models leads Markov chain Monte Carlo (MCMC) methods to suffer from the so-called label switching problem. In this article, we treat the corresponding MCMC outputs using a recent approach, namely, the Equivalence Classes Representative (ECR) algorithm and conclude that it can effectively solve the label switching problem by considering several examples of LCMs, such as mixtures of regressions, hidden Markov models, and Markov random fields. Moreover, the superiority of this method over other approaches becomes apparent.     

Variational Bayes for estimating the parameters of a hidden Potts model

Hidden Markov random field models provide an appealing representation of images and other spatial problems. The drawback is that inference is not straightforward for these models as the normalisation constant for the likelihood is generally intractable except for very small observation sets. Variational methods are an emerging tool for Bayesian inference and they have already been successfully applied in other contexts. Focusing on the particular case of a hidden Potts model with Gaussian noise, we show how variational Bayesian methods can be applied to hidden Markov random field inference. To tackle the obstacle of the intractable normalising constant for the likelihood, we explore alternative estimation approaches for incorporation into the variational Bayes algorithm. We consider a pseudo-likelihood approach as well as the more recent reduced dependence approximation of the normalisation constant. To illustrate the effectiveness of these approaches we present empirical results from the analysis of simulated datasets. We also analyse a real dataset and compare results with those of previous analyses as well as those obtained from the recently developed auxiliary variable MCMC method and the recursive MCMC method. Our results show that the variational Bayesian analyses can be carried out much faster than the MCMC analyses and produce good estimates of model parameters. We also found that the reduced dependence approximation of the normalisation constant outperformed the pseudo-likelihood approximation in our analysis of real and synthetic datasets.     

A Conditional Autoregressive Gaussian Process for Irregularly Spaced Multivariate Data with Application to Modelling Large Sets of Binary Data

A Gaussian conditional autoregressive (CAR) formulation is presented that permits the modelling of the spatial dependence and the dependence between multivariate random variables at irregularly spaced sites so capturing some of the modelling advantages of the geostatistical approach. The model benefits not only from the explicit availability of the full conditionals but also from the computational simplicity of the precision matrix determinant calculation using a closed form expression involving the eigenvalues of a precision matrix submatrix. The introduction of covariates into the model adds little computational complexity to the analysis and thus the method can be straightforwardly extended to regression models. The model, because of its computational simplicity, is well suited to application involving the fully Bayesian analysis of large data sets involving multivariate measurements with a spatial ordering. An extension to spatio-temporal data is also considered. Here, we demonstrate use of the model in the analysis of bivariate binary data where the observed data is modelled as the sign of the hidden CAR process. A case study involving over 450 irregularly spaced sites and the presence or absence of each of two species of rain forest trees at each site is presented; Markov chain Monte Carlo (MCMC) methods are implemented to obtain posterior distributions of all unknowns. The MCMC method works well with simulated data and the tree biodiversity data set.     

On Block Updating in Markov Random Field Models for Disease Mapping

Gaussian Markov random field (GMRF) models are commonly used to model spatial correlation in disease mapping applications. For Bayesian inference by MCMC, so far mainly single-site updating algorithms have been considered. However, convergence and mixing properties of such algorithms can be extremely poor due to strong dependencies of parameters in the posterior distribution. In this paper, we propose various block sampling algorithms in order to improve the MCMC performance. The methodology is rather general, allows for non-standard full conditionals, and can be applied in a modular fashion in a large number of different scenarios. For illustration we consider three different applications: two formulations for spatial modelling of a single disease (with and without additional unstructured parameters respectively), and one formulation for the joint analysis of two diseases. The results indicate that the largest benefits are obtained if parameters and the corresponding hyperparameter are updated jointly in one large block. Implementation of such block algorithms is relatively easy using methods for fast sampling of Gaussian Markov random fields ( Rue, 2001 ). By comparison, Monte Carlo estimates based on single-site updating can be rather misleading, even for very long runs. Our results may have wider relevance for efficient MCMC simulation in hierarchical models with Markov random field components.     

C219. Comment on ”calculating binomial probabilities when the trial probabilities are unequal“

In complex models like hidden Markov chains, the convergence of the MCMC algorithms used to approximate the posterior distribution and the Bayes estimates of the parameters of interest must be controlled in a robust manner. We propose in this paper a series of online controls, which rely on classical non-parametric tests, to evaluate independence from the start-up distribution, stability of the Markov chain, and asymptotic normality. These tests lead to graphical control spreadsheets which arepresentedin the set-up of normalmixture hidden Markov chains to compare the full Gibbs sampler with an aggregated Gibbs sampler based on the forward – backward formulas.     

A two-state regime switching autoregressive model with an application to river flow analysis

We propose a regime switching autoregressive model and apply it to analyze daily water discharge series of River Tisza in Hungary. The dynamics is governed by two regimes, along which both the autoregressive coefficients and the innovation distributions are altering, moreover, the hidden regime indicator process is allowed to be non-Markovian. After examining stationarity and basic properties of the model, we turn to its estimation by Markov Chain Monte Carlo (MCMC) methods and propose two algorithms. The values of the latent process serve as auxiliary parameters in the first one, while the change points of the regimes do the same in the second one in a reversible jump MCMC setting. After comparing the mixing performance of the two methods, the model is fitted to the water discharge data. Simulations show that it reproduces the important features of the water discharge series such as the highly skewed marginal distribution and the asymmetric shape of the hydrograph.     

Forecasting with non-homogeneous hidden Markov models

We present a Bayesian forecasting methodology of discrete-time finite state-space hidden Markov models with non-constant transition matrix that depends on a set of exogenous covariates. We describe an MCMC reversible jump algorithm for predictive inference, allowing for model uncertainty regarding the set of covariates that affect the transition matrix. We apply our models to interest rates and we show that our general model formulation improves the predictive ability of standard homogeneous hidden Markov models.     

Bayesian clustering for continuous-time hidden Markov models

We develop clustering procedures for longitudinal trajectories based on a continuous-time hidden Markov model (CTHMM) and a generalized linear observation model. Specifically, in this article we carry out finite and infinite mixture model-based clustering for a CTHMM and achieve inference using Markov chain Monte Carlo (MCMC). For a finite mixture model with a prior on the number of components, we implement reversible-jump MCMC to facilitate the trans-dimensional move between models with different numbers of clusters. For a Dirichlet process mixture model, we utilize restricted Gibbs sampling split–merge proposals to improve the performance of the MCMC algorithm. We apply our proposed algorithms to simulated data as well as a real-data example, and the results demonstrate the desired performance of the new sampler.     

Parallel algorithms for Markov chain Monte Carlo methods in latent spatial Gaussian models

Markov chain Monte Carlo (MCMC) implementations of Bayesian inference for latent spatial Gaussian models are very computationally intensive, and restrictions on storage and computation time are limiting their application to large problems. Here we propose various parallel MCMC algorithms for such models. The algorithms' performance is discussed with respect to a simulation study, which demonstrates the increase in speed with which the algorithms explore the posterior distribution as a function of the number of processors. We also discuss how feasible problem size is increased by use of these algorithms.     

A geometric approach to transdimensional markov chain monte carlo

The authors present theoretical results that show how one can simulate a mixture distribution whose components live in subspaces of different dimension by reformulating the problem in such a way that observations may be drawn from an auxiliary continuous distribution on the largest subspace and then transformed in an appropriate fashion. Motivated by the importance of enlarging the set of available Markov chain Monte Carlo (MCMC) techniques, the authors show how their results can be fruitfully employed in problems such as model selection (or averaging) of nested models, or regeneration of Markov chains for evaluating standard deviations of estimated expectations derived from MCMC simulations.     

Spatial Modeling of Habitat Preferences of Biological Species using Markov Random Fields

Spatial modeling has gained interest in ecology during the past two decades, especially in the area of biodiversity, where reliable distribution maps are required. Several methods have been proposed to construct distribution maps, most of them acknowledging the presence of spatial interactions. In many cases, a key problem is the lack of true absence data. We present here a model suitable for use when true absence data are missing. The quality of the estimates obtained from the model is evaluated using ROC curve analysis as well as a quadratic cost function, computed from the false positive and false negative error rates. The model is also tested under random and clustered scattering of the presence records. We also present an application of the model to the construction of distribution maps of two endemic bird species in México.     

Regenerative Markov Chain Importance Sampling

We introduce Markov Chain Importance Sampling (MCIS), which combines importance sampling (IS) and Markov Chain Monte Carlo (MCMC) to estimate some characteristics of a non-normalized multi-dimensional distribution. Especially, we introduce some importance functions whose variates are regeneratively generated by MCMC; these variates then are used to estimate the quantity of interest through IS. Because MCIS is regenerative, it overcomes the burn-in problem associated with MCMC. It could also speed up the mixing rate in MCMC.     

Double Generalized Spatial Econometric Models

The authors offer a unified method extending traditional spatial dependence with normally distributed error terms to a new class of spatial models based on the biparametric exponential family of distributions. Joint modeling of the mean and variance (or precision) parameters is proposed in this family of distributions, including spatial correlation. The proposed models are applied for analyzing Colombian land concentration, assuming that the variable of interest follows normal, gamma, and beta distributions. In all cases, the models were fitted using Bayesian methodology with the Markov Chain Monte Carlo (MCMC) algorithm for sampling from joint posterior distribution of the model parameters.     

MCMC implementation for Bayesian hidden semi-Markov models with illustrative applications

Hidden Markov models (HMMs) are flexible, well-established models useful in a diverse range of applications. However, one potential limitation of such models lies in their inability to explicitly structure the holding times of each hidden state. Hidden semi-Markov models (HSMMs) are more useful in the latter respect as they incorporate additional temporal structure by explicit modelling of the holding times. However, HSMMs have generally received less attention in the literature, mainly due to their intensive computational requirements. Here a Bayesian implementation of HSMMs is presented. Recursive algorithms are proposed in conjunction with Metropolis-Hastings in such a way as to avoid sampling from the distribution of the hidden state sequence in the MCMC sampler. This provides a computationally tractable estimation framework for HSMMs avoiding the limitations associated with the conventional EM algorithm regarding model flexibility. Performance of the proposed implementation is demonstrated through simulation experiments as well as an illustrative application relating to recurrent failures in a network of underground water pipes where random effects are also included into the HSMM to allow for pipe heterogeneity.     

Binary probability maps using a hidden conditional autoregressive Gaussian process with an application to Finnish common toad data

The Finnish common toad data of Heikkinen and Hogmander are reanalysed using an alternative fully Bayesian model that does not require a pseudolikelihood approximation and an alternative prior distribution for the true presence or absence status of toads in each 10 km×10 km square. Markov chain Monte Carlo methods are used to obtain posterior probability estimates of the square-specific presences of the common toad and these are presented as a map. The results are different from those of Heikkinen and Hogmander and we offer an explanation in terms of the prior used for square-specific presence of the toads. We suggest that our approach is more faithful to the data and avoids unnecessary confounding of effects. We demonstrate how to extend our model efficiently with square-specific covariates and illustrate this by introducing deterministic spatial changes.     

Bayesian analysis for heteroscedastic normal-Pareto mixture model with application

ABSTRACT

In this article, we propose a new distribution by mixing normal and Pareto distributions, and the new distribution provides an unusual hazard function. We model the mean and the variance with covariates for heterogeneity. Estimation of the parameters is obtained by the Bayesian method using Markov Chain Monte Carlo (MCMC) algorithms. Proposal distribution in MCMC is proposed with a defined working variable related to the observations. Through the simulation, the method shows a dependable performance of the model. We demonstrate through establishing model under a real dataset that the proposed model and method can be more suitable than the previous report.     

Markov chain Monte Carlo methods for parameter estimation of the modified Weibull distribution

In this paper, the Markov chain Monte Carlo (MCMC) method is used to estimate the parameters of a modified Weibull distribution based on a complete sample. While maximum-likelihood estimation (MLE) is the most used method for parameter estimation, MCMC has recently emerged as a good alternative. When applied to parameter estimation, MCMC methods have been shown to be easy to implement computationally, the estimates always exist and are statistically consistent, and their probability intervals are convenient to construct. Details of applying MCMC to parameter estimation for the modified Weibull model are elaborated and a numerical example is presented to illustrate the methods of inference discussed in this paper. To compare MCMC with MLE, a simulation study is provided, and the differences between the estimates obtained by the two algorithms are examined.     

Assessing convergence diagnostic tests for Bayesian Cox regression

The Markov chain Monte Carlo (MCMC) method generates samples from the posterior distribution and uses these samples to approximate expectations of quantities of interest. For the process, researchers have to decide whether the Markov chain has reached the desired posterior distribution. Using convergence diagnostic tests are very important to decide whether the Markov chain has reached the target distribution. Our interest in this study was to compare the performances of convergence diagnostic tests for all parameters of Bayesian Cox regression model with different number of iterations by using a simulation and a real lung cancer dataset.     

Collapsing of Non‐centred Parameterized MCMC Algorithms with Applications to Epidemic Models

Data augmentation is required for the implementation of many Markov chain Monte Carlo (MCMC) algorithms. The inclusion of augmented data can often lead to conditional distributions from well‐known probability distributions for some of the parameters in the model. In such cases, collapsing (integrating out parameters) has been shown to improve the performance of MCMC algorithms. We show how integrating out the infection rate parameter in epidemic models leads to efficient MCMC algorithms for two very different epidemic scenarios, final outcome data from a multitype SIR epidemic and longitudinal data from a spatial SI epidemic. The resulting MCMC algorithms give fresh insight into real‐life epidemic data sets.     

Free energy methods for Bayesian inference: efficient exploration of univariate Gaussian mixture posteriors

Because of their multimodality, mixture posterior distributions are difficult to sample with standard Markov chain Monte Carlo (MCMC) methods. We propose a strategy to enhance the sampling of MCMC in this context, using a biasing procedure which originates from computational Statistical Physics. The principle is first to choose a “reaction coordinate”, that is, a “direction” in which the target distribution is multimodal. In a second step, the marginal log-density of the reaction coordinate with respect to the posterior distribution is estimated; minus this quantity is called “free energy” in the computational Statistical Physics literature. To this end, we use adaptive biasing Markov chain algorithms which adapt their targeted invariant distribution on the fly, in order to overcome sampling barriers along the chosen reaction coordinate. Finally, we perform an importance sampling step in order to remove the bias and recover the true posterior. The efficiency factor of the importance sampling step can easily be estimated a priori once the bias is known, and appears to be rather large for the test cases we considered.