Filtering via approximate Bayesian computation

Approximate Bayesian computation (ABC) has become a popular technique to facilitate Bayesian inference from complex models. In this article we present an ABC approximation designed to perform biased filtering for a Hidden Markov Model when the likelihood function is intractable. We use a sequential Monte Carlo (SMC) algorithm to both fit and sample from our ABC approximation of the target probability density. This approach is shown to, empirically, be more accurate w.r.t.?the original filter than competing methods. The theoretical bias of our method is investigated; it is shown that the bias goes to zero at the expense of increased computational effort. Our approach is illustrated on a constrained sequential lasso for portfolio allocation to 15 constituents of the FTSE 100 share index.     

Long-range dependence and approximate Bayesian computation

In this work, we propose a method for estimating the Hurst index, or memory parameter, of a stationary process with long memory in a Bayesian fashion. Such approach provides an approximation for the posterior distribution for the memory parameter and it is based on a simple application of the so-called approximate Bayesian computation (ABC), also known as likelihood-free method. Some popular existing estimators are reviewed and compared to this method for the fractional Brownian motion, for a long-range binary process and for the Rosenblatt process. The performance of our proposal is remarkably efficient.     

Generalized multiple-point Metropolis algorithms for approximate Bayesian computation

It is well known that the approximate Bayesian computation algorithm based on Markov chain Monte Carlo methods suffers from the sensitivity to the choice of starting values, inefficiency and a low acceptance rate. To overcome these problems, this study proposes a generalization of the multiple-point Metropolis algorithm, which proceeds by generating multiple-dependent proposals and then by selecting a candidate among the set of proposals on the basis of weights that can be chosen arbitrarily. The performance of the proposed algorithm is illustrated by using both simulated and real data.     

Fast approximate Bayesian computation for estimating parameters in differential equations

Approximate Bayesian computation (ABC) using a sequential Monte Carlo method provides a comprehensive platform for parameter estimation, model selection and sensitivity analysis in differential equations. However, this method, like other Monte Carlo methods, incurs a significant computational cost as it requires explicit numerical integration of differential equations to carry out inference. In this paper we propose a novel method for circumventing the requirement of explicit integration by using derivatives of Gaussian processes to smooth the observations from which parameters are estimated. We evaluate our methods using synthetic data generated from model biological systems described by ordinary and delay differential equations. Upon comparing the performance of our method to existing ABC techniques, we demonstrate that it produces comparably reliable parameter estimates at a significantly reduced execution time.     

Reverse engineering gene regulatory networks using approximate Bayesian computation

Gene regulatory networks are collections of genes that interact with one other and with other substances in the cell. By measuring gene expression over time using high-throughput technologies, it may be possible to reverse engineer, or infer, the structure of the gene network involved in a particular cellular process. These gene expression data typically have a high dimensionality and a limited number of biological replicates and time points. Due to these issues and the complexity of biological systems, the problem of reverse engineering networks from gene expression data demands a specialized suite of statistical tools and methodologies. We propose a non-standard adaptation of a simulation-based approach known as Approximate Bayesian Computing based on Markov chain Monte Carlo sampling. This approach is particularly well suited for the inference of gene regulatory networks from longitudinal data. The performance of this approach is investigated via simulations and using longitudinal expression data from a genetic repair system in Escherichia coli.     

A rare event approach to high-dimensional approximate Bayesian computation

Approximate Bayesian computation (ABC) methods permit approximate inference for intractable likelihoods when it is possible to simulate from the model. However, they perform poorly for high-dimensional data and in practice must usually be used in conjunction with dimension reduction methods, resulting in a loss of accuracy which is hard to quantify or control. We propose a new ABC method for high-dimensional data based on rare event methods which we refer to as RE-ABC. This uses a latent variable representation of the model. For a given parameter value, we estimate the probability of the rare event that the latent variables correspond to data roughly consistent with the observations. This is performed using sequential Monte Carlo and slice sampling to systematically search the space of latent variables. In contrast, standard ABC can be viewed as using a more naive Monte Carlo estimate. We use our rare event probability estimator as a likelihood estimate within the pseudo-marginal Metropolis–Hastings algorithm for parameter inference. We provide asymptotics showing that RE-ABC has a lower computational cost for high-dimensional data than standard ABC methods. We also illustrate our approach empirically, on a Gaussian distribution and an application in infectious disease modelling.     

Diagnostic tools for approximate Bayesian computation using the coverage property

Approximate Bayesian computation (ABC) is an approach to sampling from an approximate posterior distribution in the presence of a computationally intractable likelihood function. A common implementation is based on simulating model, parameter and dataset triples from the prior, and then accepting as samples from the approximate posterior, those model and parameter pairs for which the corresponding dataset, or a summary of that dataset, is ‘close’ to the observed data. Closeness is typically determined though a distance measure and a kernel scale parameter. Appropriate choice of that parameter is important in producing a good quality approximation. This paper proposes diagnostic tools for the choice of the kernel scale parameter based on assessing the coverage property, which asserts that credible intervals have the correct coverage levels in appropriately designed simulation settings. We provide theoretical results on coverage for both model and parameter inference, and adapt these into diagnostics for the ABC context. We re‐analyse a study on human demographic history to determine whether the adopted posterior approximation was appropriate. Code implementing the proposed methodology is freely available in the R package abctools .     

An adaptive sequential Monte Carlo method for approximate Bayesian computation

Approximate Bayesian computation (ABC) is a popular approach to address inference problems where the likelihood function is intractable, or expensive to calculate. To improve over Markov chain Monte Carlo (MCMC) implementations of ABC, the use of sequential Monte Carlo (SMC) methods has recently been suggested. Most effective SMC algorithms that are currently available for ABC have a computational complexity that is quadratic in the number of Monte Carlo samples (Beaumont et al., Biometrika 86:983?C990, 2009; Peters et al., Technical report, 2008; Toni et al., J.?Roy. Soc. Interface 6:187?C202, 2009) and require the careful choice of simulation parameters. In this article an adaptive SMC algorithm is proposed which admits a computational complexity that is linear in the number of samples and adaptively determines the simulation parameters. We demonstrate our algorithm on a toy example and on a birth-death-mutation model arising in epidemiology.     

Choosing summary statistics by least angle regression for approximate Bayesian computation

Bayesian statistical inference relies on the posterior distribution. Depending on the model, the posterior can be more or less difficult to derive. In recent years, there has been a lot of interest in complex settings where the likelihood is analytically intractable. In such situations, approximate Bayesian computation (ABC) provides an attractive way of carrying out Bayesian inference. For obtaining reliable posterior estimates however, it is important to keep the approximation errors small in ABC. The choice of an appropriate set of summary statistics plays a crucial role in this effort. Here, we report the development of a new algorithm that is based on least angle regression for choosing summary statistics. In two population genetic examples, the performance of the new algorithm is better than a previously proposed approach that uses partial least squares.     

A new approach to choose acceptance cutoff for approximate Bayesian computation

The approximate Bayesian computation (ABC) algorithm is used to estimate parameters from complicated phenomena, where likelihood is intractable. Here, we report the development of an algorithm to choose the tolerance level for ABC. We have illustrated the performance of our proposed method by simulating the estimation of scaled mutation and recombination rates. The result shows that the proposed algorithm performs well.     

On sequential Monte Carlo, partial rejection control and approximate Bayesian computation

We present a variant of the sequential Monte Carlo sampler by incorporating the partial rejection control mechanism of Liu (2001). We show that the resulting algorithm can be considered as a sequential Monte Carlo sampler with a modified mutation kernel. We prove that the new sampler can reduce the variance of the incremental importance weights when compared with standard sequential Monte Carlo samplers, and provide a central limit theorem. Finally, the sampler is adapted for application under the challenging approximate Bayesian computation modelling framework.     

Accelerating inference for diffusions observed with measurement error and large sample sizes using approximate Bayesian computation

In recent years, dynamical modelling has been provided with a range of breakthrough methods to perform exact Bayesian inference. However, it is often computationally unfeasible to apply exact statistical methodologies in the context of large data sets and complex models. This paper considers a nonlinear stochastic differential equation model observed with correlated measurement errors and an application to protein folding modelling. An approximate Bayesian computation (ABC)-MCMC algorithm is suggested to allow inference for model parameters within reasonable time constraints. The ABC algorithm uses simulations of ‘subsamples’ from the assumed data-generating model as well as a so-called ‘early-rejection’ strategy to speed up computations in the ABC-MCMC sampler. Using a considerate amount of subsamples does not seem to degrade the quality of the inferential results for the considered applications. A simulation study is conducted to compare our strategy with exact Bayesian inference, the latter resulting two orders of magnitude slower than ABC-MCMC for the considered set-up. Finally, the ABC algorithm is applied to a large size protein data. The suggested methodology is fairly general and not limited to the exemplified model and data.     

Bayesian computation for Log-Gaussian Cox processes: a comparative analysis of methods

The Log-Gaussian Cox process is a commonly used model for the analysis of spatial point pattern data. Fitting this model is difficult because of its doubly stochastic property, that is, it is a hierarchical combination of a Poisson process at the first level and a Gaussian process at the second level. Various methods have been proposed to estimate such a process, including traditional likelihood-based approaches as well as Bayesian methods. We focus here on Bayesian methods and several approaches that have been considered for model fitting within this framework, including Hamiltonian Monte Carlo, the Integrated nested Laplace approximation, and Variational Bayes. We consider these approaches and make comparisons with respect to statistical and computational efficiency. These comparisons are made through several simulation studies as well as through two applications, the first examining ecological data and the second involving neuroimaging data.     

An approximate Bayesian up-down method for estimating a percentage point on a dose-response curve

While the up-down method for estimating a percentage point on a dose-response curve has received considerable attention, a general Bayesian solution to the up-down design and estimation has never been presented, probably due to its computational complexity both in design and use. This paper presents a theoretical approach for up-down experimental designs with unknown location and slope parameters, and a practical approach for their use. The simplex method is used to find the optimal starting dose level and step sizes that minimize the expected root mean square error for a fixed number of observations and a reduced number of step sizes. The Bayesian estimate is then approximated by a polynomial formula. The coefficients of the formula are also chosen using simplex minimization. Two example solutions are given with uniform-uniform and normal-gamma joint prior distributions, showing that the simplifying assumptions make the method far easier to use with only a marginal increase in expected root mean square error. We show how to adapt these prior distributions to a wide range of frequently encountered applications.     

Bayesian computation for the superposition of nonhomogeneous poisson processes

Bayesian inference for the superposition of nonhomogeneous Poisson processes is studied. A Markov-chain Monte Carlo method with data augmentation is developed to compute the features of the posterior distribution. For each observed failure epoch, a latent variable is introduced that indicates which component of the superposition model gives rise to the failure. This data-augmentation approach facilitates specification of the transitional kernel in the Markov chain. Moreover, new Bayesian tests are developed for the full superposition model against simpler submodels. Model determination by a predictive likelihood approach is studied. A numerical example based on a real data set is given.     

Application of a predictive distribution formula to Bayesian computation for incomplete data models

We consider exact and approximate Bayesian computation in the presence of latent variables or missing data. Specifically we explore the application of a posterior predictive distribution formula derived in Sweeting And Kharroubi (2003), which is a particular form of Laplace approximation, both as an importance function and a proposal distribution. We show that this formula provides a stable importance function for use within poor man’s data augmentation schemes and that it can also be used as a proposal distribution within a Metropolis-Hastings algorithm for models that are not analytically tractable. We illustrate both uses in the case of a censored regression model and a normal hierarchical model, with both normal and Student t distributed random effects. Although the predictive distribution formula is motivated by regular asymptotic theory, it is not necessary that the likelihood has a closed form or that it possesses a local maximum.     

Predictive sets with approximate frequentist and Bayesian validity for arbitrary priors

With a view to predicting a scalar-valued future observation on the basis of past observations, we explore predictive sets having frequentist as well as Bayesian validity for arbitrary priors in a higher-order asymptotic sense. It is found that a connection with locally unbiased tests is useful for this purpose. Illustrative examples are given. Computation and simulation studies lend support to our asymptotic results in finite samples. The issue of expected lengths of our predictive sets is also discussed.     

Bayesian and approximate bayesian solutions to simultaneous estimation of multiple dynamic processes

In this paper we motivate solutions to simultaneous estimation of multiple dynamic processes in situations where the correspondence between the set of measurements and the set of processes is uncertain and thus special modelling is required to accomodate the unclassified data. We derive the optimal Bayesian solution for non linear processes which turns out to be very computationally complicated, and then suggest a quasi Bayes approximation which removes the complication due to the uncertain measurement-process correspondence. Numerical illustrations are provided for the linear case.     

The use of restricted cubic splines to approximate complex hazard functions in the analysis of time-to-event data: a simulation study

If interest lies in reporting absolute measures of risk from time-to-event data then obtaining an appropriate approximation to the shape of the underlying hazard function is vital. It has previously been shown that restricted cubic splines can be used to approximate complex hazard functions in the context of time-to-event data. The degree of complexity for the spline functions is dictated by the number of knots that are defined. We highlight through the use of a motivating example that complex hazard function shapes are often required when analysing time-to-event data. Through the use of simulation, we show that provided a sufficient number of knots are used, the approximated hazard functions given by restricted cubic splines fit closely to the true function for a range of complex hazard shapes. The simulation results also highlight the insensitivity of the estimated relative effects (hazard ratios) to the correct specification of the baseline hazard.