On directional Metropolis–Hastings algorithms

New Metropolis–Hastings algorithms using directional updates are introduced in this paper. Each iteration of a directional Metropolis–Hastings algorithm consists of three steps (i) generate a line by sampling an auxiliary variable, (ii) propose a new state along the line, and (iii) accept/reject according to the Metropolis–Hastings acceptance probability. We consider two classes of directional updates. The first uses a point in ⁿ as auxiliary variable, the second an auxiliary direction vector. The proposed algorithms generalize previous directional updating schemes since we allow the distribution of the auxiliary variable to depend on properties of the target at the current state. By letting the proposal distribution along the line depend on the density of the auxiliary variable, we identify proposal mechanisms that give unit acceptance rate. When we use direction vector as auxiliary variable, we get the advantageous effect of large moves in the Markov chain and hence the autocorrelation length of the samples is small. We apply the directional Metropolis–Hastings algorithms to a Gaussian example, a mixture of Gaussian densities, and a Bayesian model for seismic data.     

On adaptive Metropolis–Hastings methods

This paper presents a method for adaptation in Metropolis–Hastings algorithms. A product of a proposal density and K copies of the target density is used to define a joint density which is sampled by a Gibbs sampler including a Metropolis step. This provides a framework for adaptation since the current value of all K copies of the target distribution can be used in the proposal distribution. The methodology is justified by standard Gibbs sampling theory and generalizes several previously proposed algorithms. It is particularly suited to Metropolis-within-Gibbs updating and we discuss the application of our methods in this context. The method is illustrated with both a Metropolis–Hastings independence sampler and a Metropolis-with-Gibbs independence sampler. Comparisons are made with standard adaptive Metropolis–Hastings methods.     

Variance reduction of estimators arising from Metropolis–Hastings algorithms

The Metropolis–Hastings algorithm is one of the most basic and well-studied Markov chain Monte Carlo methods. It generates a Markov chain which has as limit distribution the target distribution by simulating observations from a different proposal distribution. A proposed value is accepted with some particular probability otherwise the previous value is repeated. As a consequence, the accepted values are repeated a positive number of times and thus any resulting ergodic mean is, in fact, a weighted average. It turns out that this weighted average is an importance sampling-type estimator with random weights. By the standard theory of importance sampling, replacement of these random weights by their (conditional) expectations leads to more efficient estimators. In this paper we study the estimator arising by replacing the random weights with certain estimators of their conditional expectations. We illustrate by simulations that it is often more efficient than the original estimator while in the case of the independence Metropolis–Hastings and for distributions with finite support we formally prove that it is even better than the “optimal” importance sampling estimator.     

Pseudo-perfect and adaptive variants of the Metropolis–Hastings algorithm with an independent candidate density

We describe and examine an imperfect variant of a perfect sampling algorithm based on the Metropolis–Hastings algorithm that appears to perform better than a more traditional approach in terms of speed and accuracy. We then describe and examine an ‘adaptive’ Metropolis–Hastings algorithm which generates and updates a self-target candidate density in such a way that there is no ‘wrong choice’ for an initial candidate density. Simulation examples are provided.     

Adaptive optimal scaling of Metropolis–Hastings algorithms using the Robbins–Monro process

ABSTRACT

We present an adaptive method for the automatic scaling of random-walk Metropolis–Hastings algorithms, which quickly and robustly identifies the scaling factor that yields a specified overall sampler acceptance probability. Our method relies on the use of the Robbins–Monro search process, whose performance is determined by an unknown steplength constant. Based on theoretical considerations we give a simple estimator of this constant for Gaussian proposal distributions. The effectiveness of our method is demonstrated with both simulated and real data examples.     

Stability of noisy Metropolis–Hastings

Pseudo-marginal Markov chain Monte Carlo methods for sampling from intractable distributions have gained recent interest and have been theoretically studied in considerable depth. Their main appeal is that they are exact, in the sense that they target marginally the correct invariant distribution. However, the pseudo-marginal Markov chain can exhibit poor mixing and slow convergence towards its target. As an alternative, a subtly different Markov chain can be simulated, where better mixing is possible but the exactness property is sacrificed. This is the noisy algorithm, initially conceptualised as Monte Carlo within Metropolis, which has also been studied but to a lesser extent. The present article provides a further characterisation of the noisy algorithm, with a focus on fundamental stability properties like positive recurrence and geometric ergodicity. Sufficient conditions for inheriting geometric ergodicity from a standard Metropolis–Hastings chain are given, as well as convergence of the invariant distribution towards the true target distribution.     

A double Metropolis–Hastings sampler for spatial models with intractable normalizing constants

The problem of simulating from distributions with intractable normalizing constants has received much attention in recent literature. In this article, we propose an asymptotic algorithm, the so-called double Metropolis–Hastings (MH) sampler, for tackling this problem. Unlike other auxiliary variable algorithms, the double MH sampler removes the need for exact sampling, the auxiliary variables being generated using MH kernels, and thus can be applied to a wide range of problems for which exact sampling is not available. For the problems for which exact sampling is available, it can typically produce the same accurate results as the exchange algorithm, but using much less CPU time. The new method is illustrated by various spatial models.     

Augmented pseudo-marginal Metropolis–Hastings for partially observed diffusion processes

Statistics and Computing -     

Not every Gibbs sampler is a special case of the Metropolis–Hastings algorithm

It is commonly asserted that the Gibbs sampler is a special case of the Metropolis–Hastings (MH) algorithm. While this statement is true for certain Gibbs samplers, it is not true in general for the version that is taught and used most often, namely, the deterministic scan Gibbs sampler. In this note, I prove that that there exist deterministic scan Gibbs samplers that do not exhibit detailed balance and hence cannot be considered MH samplers. The nuances of various Gibbs sampling schemes are discussed.     

A modified adaptive accept–reject algorithm for univariate densities with bounded support

The need to simulate from a univariate density arises in several settings, particularly in Bayesian analysis. An especially efficient algorithm which can be used to sample from a univariate density, f _X , is the adaptive accept–reject algorithm. To implement the adaptive accept–reject algorithm, the user has to envelope T ° f _X , where T is some transformation such that the density g(x) ∝ T ^?1 (α+β x) is easy to sample from. Successfully enveloping T ° f _X , however, requires that the user identify the number and location of T ° f _X ’s inflection points. This is not always a trivial task. In this paper, we propose an adaptive accept–reject algorithm which relieves the user of precisely identifying the location of T ° f _X ’s inflection points. This new algorithm is shown to be efficient and can be used to sample from any density such that its support is bounded and its log is three-times differentiable.     

A continuously adaptive nonparametric two–sample test

A two–sample test statistic for detecting shifts in location is developed for a broad range of underlying distributions using adaptive techniques. The test statistic is a linear rank statistics which uses a simple modification of the Wilcoxon test; the scores are Winsorized ranks where the upper and lower Winsorinzing proportions are estimated in the first stage of the adaptive procedure using sample the first stage of the adaptive procedure using sample measures of the distribution's skewness and tailweight. An empirical relationship between the Winsorizing proportions and the sample skewness and tailweight allows for a ‘continuous’ adaptation of the test statistic to the data. The test has good asymptotic properties, and the small sample results are compared with other populatr parametric, nonparametric, and two–stage tests using Monte Carlo methods. Based on these results, this proposed test procedure is recommended for moderate and larger sample sizes.     

Statistical analysis of competing risks model from Marshall–Olkin extended Chen distribution under adaptive progressively interval censoring with random removals

In this paper, a new censoring scheme named by adaptive progressively interval censoring scheme is introduced. The competing risks data come from Marshall–Olkin extended Chen distribution under the new censoring scheme with random removals. We obtain the maximum likelihood estimators of the unknown parameters and the reliability function by using the EM algorithm based on the failure data. In addition, the bootstrap percentile confidence intervals and bootstrap-t confidence intervals of the unknown parameters are obtained. To test the equality of the competing risks model, the likelihood ratio tests are performed. Then, Monte Carlo simulation is conducted to evaluate the performance of the estimators under different sample sizes and removal schemes. Finally, a real data set is analyzed for illustration purpose.     

Adaptive PORT–MVRB estimation: an empirical comparison of two heuristic algorithms

In this article, we deal with an empirical comparison of two data-driven heuristic procedures of estimation of a positive extreme value index (EVI), working thus with heavy right tails. The semi-parametric EVI-estimators under consideration, the so-called peaks over random threshold (PORT)–minimum-variance reduced-bias (MVRB) EVI-estimators, are location and scale-invariant estimators, based on the PORT methodology applied to second-order MVRB EVI-estimators. Trivial adaptations of these algorithms make them work for a similar estimation of other parameters of extreme events, such as the Value-at-Risk at a level p, the expected shortfall and the probability of exceedance of a high level x, among others. Applications to simulated data sets and to real data sets in the field of finance are provided.     

On the effect of inducted negative correlation rate for beta acceptance–rejection algorithms

One of the variance reduction methods in simulation experiments is negative correlation induction, and in particular the use of the antithetic variates. The simultaneous use of antithetic variates and an acceptance–rejection method has been studied in some papers, where the inducted negative correlation has been calculated. In this study, the factors affecting the inducted negative correlation rate are addressed. To do this, the beta distribution is first selected to generate negatively correlated random variates using the acceptance–rejection method. The effects of both the efficiency of the acceptance–rejection method and the initial negative correlation rate on the inducted negative correlation are explored. Results show that both factors have significant effects; therefore, a combination of both can lead to algorithms better able to generate negative correlations.     

Marshall–Olkin gamma–Weibull distribution with applications

Abstract

A Marshall–Olkin variant of the Provost type gamma–Weibull probability distribution is being introduced in this paper. Some of its statistical functions and numerical characteristics among others characteristics function, moment generalizing function, central moments of real order are derived in the computational series expansion form and various illustrative special cases are discussed. This density function is utilized to model two real data sets. The new distribution provides a better fit than related distributions as measured by the Anderson–Darling and Cramér–von Mises statistics. The proposed distribution could find applications for instance in the physical and biological sciences, hydrology, medicine, meteorology, engineering, etc.     

The Co-Integrated Vector Autoregression with Errors–in–Variables

The co-integrated vector autoregression is extended to allow variables to be observed with classical measurement errors (ME). For estimation, the model is parametrized as a time invariant state-space form, and an accelerated expectation-maximization algorithm is derived. A simulation study shows that (i) the finite-sample properties of the maximum likelihood (ML) estimates and reduced rank test statistics are excellent (ii) neglected measurement errors will generally distort unit root inference due to a moving average component in the residuals, and (iii) the moving average component may–in principle–be approximated by a long autoregression, but a pure autoregression cannot identify the autoregressive structure of the latent process, and the adjustment coefficients are estimated with a substantial asymptotic bias. An application to the zero-coupon yield-curve is given.     

An extension of Horvitz–Thompson estimator used in adaptive cluster sampling to continuous universe

In this paper, an extension of Horvitz–Thompson estimator used in adaptive cluster sampling to continuous universe is developed. Main new results are presented in theorems. The primary notions of discrete population are transferred to continuous population. First and second order inclusion probabilities for networks are delivered. Horvitz–Thompson estimator for adaptive cluster sampling in continuous universe is constructed. The unbiasedness of the estimator is proven. Variance and unbiased variance estimator are delivered. Finally, the theory is illustrated with an example.