A geometric approach to transdimensional markov chain monte carlo

The authors present theoretical results that show how one can simulate a mixture distribution whose components live in subspaces of different dimension by reformulating the problem in such a way that observations may be drawn from an auxiliary continuous distribution on the largest subspace and then transformed in an appropriate fashion. Motivated by the importance of enlarging the set of available Markov chain Monte Carlo (MCMC) techniques, the authors show how their results can be fruitfully employed in problems such as model selection (or averaging) of nested models, or regeneration of Markov chains for evaluating standard deviations of estimated expectations derived from MCMC simulations.     

Bayes and MCMC for Undergraduates

Students of statistics should be taught the ideas and methods that are widely used in practice and that will help them understand the world of statistics. Today, this means teaching them about Bayesian methods. In this article, I present ideas on teaching an undergraduate Bayesian course that uses Markov chain Monte Carlo and that can be a second course or, for strong students, a first course in statistics.     

Approximate Bayesian Computation for Exponential Random Graph Models for Large Social Networks

We consider the issue of sampling from the posterior distribution of exponential random graph (ERG) models and other statistical models with intractable normalizing constants. Existing methods based on exact sampling are either infeasible or require very long computing time. We study a class of approximate Markov chain Monte Carlo (MCMC) sampling schemes that deal with this issue. We also develop a new Metropolis–Hastings kernel to sample sparse large networks from ERG models. We illustrate the proposed methods on several examples.     

Bayesian inference in hidden Markov models through the reversible jump Markov chain Monte Carlo method

Hidden Markov models form an extension of mixture models which provides a flexible class of models exhibiting dependence and a possibly large degree of variability. We show how reversible jump Markov chain Monte Carlo techniques can be used to estimate the parameters as well as the number of components of a hidden Markov model in a Bayesian framework. We employ a mixture of zero-mean normal distributions as our main example and apply this model to three sets of data from finance, meteorology and geomagnetism.     

Long memory stochastic volatility : A bayesian approach

We propose a simulation-based Bayesian approach to the analysis of long memory stochastic volatility models, stationary and nonstationary. The main tool used to reduce the likelihood function to a tractable form is an approximate state-space representation of the model, A data set of stock market returns is analyzed with the proposed method. The approach taken here allows a quantitative assessment of the empirical evidence in favor of the stationarity, or nonstationarity, of the instantaneous volatility of the data.     

Multivariate process capability a bayesian perspective

In this paper an attempt has been made to examine the multivariate versions of the common process capability indices (PCI's) denoted by C_p and C_pk . Markov chain Monte Carlo (MCMC) methods are used to generate sampling distributions for the various PCI's from where inference is performed. Some Bayesian model checking techniques are developed and implemented to examine how well our model fits the data. Finally the methods are exemplified on a historical aircraft data set collected by the Pratt and Whitney Company.     

A method for approximate fully bayesian analysis of parameters

Stochastic modeling of the geology in petroleum reservoirs has become an important tool in order to investigate flow properties in the reservoir. The stochastic models used contain parameters which must be estimated based on observations and geological knowledge. The amount of data available is however quite limited due to high drilling costs etc., and the lack of data prevents the use of many of the standard data driven approaches to the parameter estimation problem. Modern simulation based methods using Markov chain Monte Carlo simulation, can however be used to do fully Bayesian analysis with respect to parameters in the reservoir model, with the drawback of relatively high computational costs. In this paper, we propose a simple, relatively fast approximate method for fully Bayesian analysis of the parameters. We illustrate the method on both simulated and real data using a two-dimensional marked point model for reservoir characterization.     

Bayesian expectile regression with asymmetric normal distribution

In this paper, we adopt the Bayesian approach to expectile regression employing a likelihood function that is based on an asymmetric normal distribution. We demonstrate that improper uniform priors for the unknown model parameters yield a proper joint posterior. Three simulated data sets were generated to evaluate the proposed method which show that Bayesian expectile regression performs well and has different characteristics comparing with Bayesian quantile regression. We also apply this approach into two real data analysis.     

A new class of multivariate skew distributions with applications to bayesian regression models

Abstract: The authors develop a new class of distributions by introducing skewness in multivariate elliptically symmetric distributions. The class, which is obtained by using transformation and conditioning, contains many standard families including the multivariate skew‐normal and t distributions. The authors obtain analytical forms of the densities and study distributional properties. They give practical applications in Bayesian regression models and results on the existence of the posterior distributions and moments under improper priors for the regression coefficients. They illustrate their methods using practical examples.     

Estimation and Properties of a Time-Varying EGARCH(1,1) in Mean Model

Time-varying GARCH-M models are commonly employed in econometrics and financial economics. Yet the recursive nature of the conditional variance makes likelihood analysis of these models computationally infeasible. This article outlines the issues and suggests to employ a Markov chain Monte Carlo algorithm which allows the calculation of a classical estimator via the simulated EM algorithm or a simulated Bayesian solution in only O(T) computational operations, where T is the sample size. Furthermore, the theoretical dynamic properties of a time-varying-parameter EGARCH(1,1)-M are derived. We discuss them and apply the suggested Bayesian estimation to three major stock markets.     

A Loss Function Model for the Restoration of Grey Level Images

Common loss functions used for the restoration of grey scale images include the zero–one loss and the sum of squared errors. The corresponding estimators, the posterior mode and the posterior marginal mean, are optimal Bayes estimators with respect to their way of measuring the loss for different error configurations. However, both these loss functions have a fundamental weakness: the loss does not depend on the spatial structure of the errors. This is important because a systematic structure in the errors can lead to misinterpretation of the estimated image. We propose a new loss function that also penalizes strong local sample covariance in the error and we discuss how the optimal Bayes estimator can be estimated using a two-step Markov chain Monte Carlo and simulated annealing algorithm. We present simulation results for some artificial data which show improvement with respect to small structures in the image.     

Adaptive multiple importance sampling for Gaussian processes

In applications of Gaussian processes (GPs) where quantification of uncertainty is a strict requirement, it is necessary to accurately characterize the posterior distribution over Gaussian process covariance parameters. This is normally done by means of standard Markov chain Monte Carlo (MCMC) algorithms, which require repeated expensive calculations involving the marginal likelihood. Motivated by the desire to avoid the inefficiencies of MCMC algorithms rejecting a considerable amount of expensive proposals, this paper develops an alternative inference framework based on adaptive multiple importance sampling (AMIS). In particular, this paper studies the application of AMIS for GPs in the case of a Gaussian likelihood, and proposes a novel pseudo-marginal-based AMIS algorithm for non-Gaussian likelihoods, where the marginal likelihood is unbiasedly estimated. The results suggest that the proposed framework outperforms MCMC-based inference of covariance parameters in a wide range of scenarios.     

Relabelling algorithms for mixture models with applications for large data sets

Mixture models are flexible tools in density estimation and classification problems. Bayesian estimation of such models typically relies on sampling from the posterior distribution using Markov chain Monte Carlo. Label switching arises because the posterior is invariant to permutations of the component parameters. Methods for dealing with label switching have been studied fairly extensively in the literature, with the most popular approaches being those based on loss functions. However, many of these algorithms turn out to be too slow in practice, and can be infeasible as the size and/or dimension of the data grow. We propose a new, computationally efficient algorithm based on a loss function interpretation, and show that it can scale up well in large data set scenarios. Then, we review earlier solutions which can scale up well for large data set, and compare their performances on simulated and real data sets. We conclude with some discussions and recommendations of all the methods studied.     

Parallelizing MCMC for Bayesian spatiotemporal geostatistical models

When MCMC methods for Bayesian spatiotemporal modeling are applied to large geostatistical problems, challenges arise as a consequence of memory requirements, computing costs, and convergence monitoring. This article describes the parallelization of a reparametrized and marginalized posterior sampling (RAMPS) algorithm, which is carefully designed to generate posterior samples efficiently. The algorithm is implemented using the Parallel Linear Algebra Package (PLAPACK). The scalability of the algorithm is investigated via simulation experiments that are implemented using a cluster with 25 processors. The usefulness of the method is illustrated with an application to sulfur dioxide concentration data from the Air Quality System database of the U.S. Environmental Protection Agency.     

Non-parametric Bayesian Estimation of a Spatial Poisson Intensity

A method introduced by Arjas & Gasbarra (1994) and later modified by Arjas & Heikkinen (1997) for the non-parametric Bayesian estimation of an intensity on the real line is generalized to cover spatial processes. The method is based on a model approximation where the approximating intensities have the structure of a piecewise constant function. Random step functions on the plane are generated using Voronoi tessellations of random point patterns. Smoothing between nearby intensity values is applied by means of a Markov random field prior in the spirit of Bayesian image analysis. The performance of the method is illustrated in examples with both real and simulated data.     

Multivariate mixtures of normals with unknown number of components

We present full Bayesian analysis of finite mixtures of multivariate normals with unknown number of components. We adopt reversible jump Markov chain Monte Carlo and we construct, in a manner similar to that of Richardson and Green (1997), split and merge moves that produce good mixing of the Markov chains. The split moves are constructed on the space of eigenvectors and eigenvalues of the current covariance matrix so that the proposed covariance matrices are positive definite. Our proposed methodology has applications in classification and discrimination as well as heterogeneity modelling. We test our algorithm with real and simulated data.     

A Hellinger distance approach to MCMC diagnostics

Bayesian analysis often requires the researcher to employ Markov Chain Monte Carlo (MCMC) techniques to draw samples from a posterior distribution which in turn is used to make inferences. Currently, several approaches to determine convergence of the chain as well as sensitivities of the resulting inferences have been developed. This work develops a Hellinger distance approach to MCMC diagnostics. An approximation to the Hellinger distance between two distributions f and g based on sampling is introduced. This approximation is studied via simulation to determine the accuracy. A criterion for using this Hellinger distance for determining chain convergence is proposed as well as a criterion for sensitivity studies. These criteria are illustrated using a dataset concerning the Anguilla australis, an eel native to New Zealand.     

R package rjmcmc: reversible jump MCMC using post‐processing

The rjmcmc package for R implements the post‐processing reversible jump Markov chain Monte Carlo (MCMC) algorithm of Barker & Link. MCMC output from each of the models is used to estimate posterior model probabilities and Bayes factors. Automatic differentiation is used to simplify implementation. The package is demonstrated on two examples.     

Random Bernstein Polynomials

Random Bernstein polynomials which are also probability distribution functions on the closed unit interval are studied. The probability law of a Bernstein polynomial so defined provides a novel prior on the space of distribution functions on [0, 1] which has full support and can easily select absolutely continuous distribution functions with a continuous and smooth derivative. In particular, the Bernstein polynomial which approximates a Dirichlet process is studied. This may be of interest in Bayesian non-parametric inference. In the second part of the paper, we study the posterior from a Bernstein–Dirichlet prior and suggest a hybrid Monte Carlo approximation of it. The proposed algorithm has some aspects of novelty since the problem under examination has a changing dimension parameter space.     

A simple method for finding a legal configuration in complex Bayesian networks

This paper deals with an important problem with large and complex Bayesian networks. Exact inference in these networks is simply not feasible owing to the huge storage requirements of exact methods. Markov chain Monte Carlo methods, however, are able to deal with these large networks but to do this they require an initial legal configuration to set off the sampler. So far nondeterministic methods such as forward sampling have often been used for this, even though the forward sampler may take an eternity to come up with a legal configuration. In this paper a novel algorithm will be presented that allows a legal configuration in a general Bayesian network to be found in polynomial time in almost all cases. The algorithm will not be proved deterministic but empirical results will demonstrate that this holds in most cases. Also, the algorithm will be justified by its simplicity and ease of implementation.