Laplace based approximate posterior inference for differential equation models

Ordinary differential equations are arguably the most popular and useful mathematical tool for describing physical and biological processes in the real world. Often, these physical and biological processes are observed with errors, in which case the most natural way to model such data is via regression where the mean function is defined by an ordinary differential equation believed to provide an understanding of the underlying process. These regression based dynamical models are called differential equation models. Parameter inference from differential equation models poses computational challenges mainly due to the fact that analytic solutions to most differential equations are not available. In this paper, we propose an approximation method for obtaining the posterior distribution of parameters in differential equation models. The approximation is done in two steps. In the first step, the solution of a differential equation is approximated by the general one-step method which is a class of numerical numerical methods for ordinary differential equations including the Euler and the Runge-Kutta procedures; in the second step, nuisance parameters are marginalized using Laplace approximation. The proposed Laplace approximated posterior gives a computationally fast alternative to the full Bayesian computational scheme (such as Makov Chain Monte Carlo) and produces more accurate and stable estimators than the popular smoothing methods (called collocation methods) based on frequentist procedures. For a theoretical support of the proposed method, we prove that the Laplace approximated posterior converges to the actual posterior under certain conditions and analyze the relation between the order of numerical error and its Laplace approximation. The proposed method is tested on simulated data sets and compared with the other existing methods.     

Bayesian penalized B-spline estimation approach for epidemic models

Ordinary differential equations (ODEs) are normally used to model dynamic processes in applied sciences such as biology, engineering, physics, and many other areas. In these models, the parameters are usually unknown, and thus they are often specified artificially or empirically. Alternatively, a feasible method is to estimate the parameters based on observed data. In this study, we propose a Bayesian penalized B-spline approach to estimate the parameters and initial values for ODEs used in epidemiology. We evaluated the efficiency of the proposed method based on simulations using the Markov chain Monte Carlo algorithm for the Kermack–McKendrick model. The proposed approach is also illustrated based on a real application to the transmission dynamics of hepatitis C virus in mainland China.     

Variational Bayes for estimating the parameters of a hidden Potts model

Hidden Markov random field models provide an appealing representation of images and other spatial problems. The drawback is that inference is not straightforward for these models as the normalisation constant for the likelihood is generally intractable except for very small observation sets. Variational methods are an emerging tool for Bayesian inference and they have already been successfully applied in other contexts. Focusing on the particular case of a hidden Potts model with Gaussian noise, we show how variational Bayesian methods can be applied to hidden Markov random field inference. To tackle the obstacle of the intractable normalising constant for the likelihood, we explore alternative estimation approaches for incorporation into the variational Bayes algorithm. We consider a pseudo-likelihood approach as well as the more recent reduced dependence approximation of the normalisation constant. To illustrate the effectiveness of these approaches we present empirical results from the analysis of simulated datasets. We also analyse a real dataset and compare results with those of previous analyses as well as those obtained from the recently developed auxiliary variable MCMC method and the recursive MCMC method. Our results show that the variational Bayesian analyses can be carried out much faster than the MCMC analyses and produce good estimates of model parameters. We also found that the reduced dependence approximation of the normalisation constant outperformed the pseudo-likelihood approximation in our analysis of real and synthetic datasets.     

Fast approximate Bayesian computation for estimating parameters in differential equations

Approximate Bayesian computation (ABC) using a sequential Monte Carlo method provides a comprehensive platform for parameter estimation, model selection and sensitivity analysis in differential equations. However, this method, like other Monte Carlo methods, incurs a significant computational cost as it requires explicit numerical integration of differential equations to carry out inference. In this paper we propose a novel method for circumventing the requirement of explicit integration by using derivatives of Gaussian processes to smooth the observations from which parameters are estimated. We evaluate our methods using synthetic data generated from model biological systems described by ordinary and delay differential equations. Upon comparing the performance of our method to existing ABC techniques, we demonstrate that it produces comparably reliable parameter estimates at a significantly reduced execution time.     

Importance sampling for signal extraction model in time series analysis

In time series analysis, signal extraction model (SEM) is used to estimate unobserved signal component from observed time series data. Since parameters of the components in SEM are often unknown in practice, a commonly used method is to estimate unobserved signal component using the maximum likelihood estimates (MLEs) of parameters of the components. This paper explores an alternative way to estimate unobserved signal component when parameters of the components are unknown. The suggested method makes use of importance sampling (IS) with Bayesian inference. The basic idea is to treat parameters of the components in SEM as a random vector and compute a posterior probability density function of the parameters using Bayesian inference. Then IS method is applied to integrate out the parameters and thus estimates of unobserved signal component, unconditional to the parameters, can be obtained. This method is illustrated with a real time series data. Then a Monte Carlo study with four different types of time series models is carried out to compare a performance of this method with that of a commonly used method. The study shows that IS method with Bayesian inference is computationally feasible and robust, and more efficient in terms of mean square errors (MSEs) than a commonly used method.     

Bayesian incidence analysis of animal tumorigenicity data

Statistical inference about tumorigenesis should focus on the tumour incidence rate. Unfortunately, in most animal carcinogenicity experiments, tumours are not observable in live animals and censoring of the tumour onset times is informative. In this paper, we propose a Bayesian method for analysing data from such studies. Our approach focuses on the incidence of tumours and accommodates occult tumours and censored onset times without restricting tumour lethality, relying on cause-of-death data, or requiring interim sacrifices. We represent the underlying state of nature by a multistate stochastic process and assume general probit models for the time-specific transition rates. These models allow the incorporation of covariates, historical control data and subjective prior information. The inherent flexibility of this approach facilitates the interpretation of results, particularly when the sample size is small or the data are sparse. We use a Gibbs sampler to estimate the relevant posterior distributions. The methods proposed are applied to data from a US National Toxicology Program carcinogenicity study.     

Approximate Bayesian Inference for Survival Models

Abstract. Bayesian analysis of time‐to‐event data, usually called survival analysis, has received increasing attention in the last years. In Cox‐type models it allows to use information from the full likelihood instead of from a partial likelihood, so that the baseline hazard function and the model parameters can be jointly estimated. In general, Bayesian methods permit a full and exact posterior inference for any parameter or predictive quantity of interest. On the other side, Bayesian inference often relies on Markov chain Monte Carlo (MCMC) techniques which, from the user point of view, may appear slow at delivering answers. In this article, we show how a new inferential tool named integrated nested Laplace approximations can be adapted and applied to many survival models making Bayesian analysis both fast and accurate without having to rely on MCMC‐based inference.     

Bayesian Analysis of DSGE Models—Some Comments

Sungbae An and Frank Schorfheide have provided an excellent review of the main elements of Bayesian inference in Dynamic Stochastic General Equilibrium (DSGE) models. Bayesian methods have, for reasons clearly outlined in the paper, a very natural role to play in DSGE analysis, and the appeal of the Bayesian paradigm is indeed strongly evidenced by the flood of empirical applications in the area over the last couple of years. We expect their paper to be the natural starting point for applied economists interested in learning about Bayesian techniques for analyzing DSGE models, and as such the paper is likely to have a strong influence on what will be considered best practice for estimating DSGE models.

The authors have, for good reasons, chosen a stylized six-equation model to present the methodology. We shall use here the large-scale model in Adolfson et al. (2005), henceforth ALLV, to illustrate a few econometric problems which we have found to be especially important as the size of the model increases. The model in ALLV is an open economy extension of the closed economy model in Christiano et al. (2005). It consists of 25 log-linearized equations, which can be written as a state space representation with 60 state variables, many of them unobserved. Fifteen observed unfiltered time series are used to estimate 51 structural parameters. An additional complication compared to the model in An and Schorfheide's paper is that some of the coefficients in the measurement equation are non-linear functions of the structural parameters. The model is currently the main vehicle for policy analysis at Sveriges Riksbank (Central Bank of Sweden) and similar models are being developed in many other policy institutions, which testifies to the model's practical relevance. The version considered here is estimated on Euro area data over the period 1980Q1-2002Q4. We refer to ALLV for details.     

Spectral density-based and measure-preserving ABC for partially observed diffusion processes. An illustration on Hamiltonian SDEs

Approximate Bayesian computation (ABC) has become one of the major tools of likelihood-free statistical inference in complex mathematical models. Simultaneously, stochastic differential equations (SDEs) have developed to an established tool for modelling time-dependent, real-world phenomena with underlying random effects. When applying ABC to stochastic models, two major difficulties arise: First, the derivation of effective summary statistics and proper distances is particularly challenging, since simulations from the stochastic process under the same parameter configuration result in different trajectories. Second, exact simulation schemes to generate trajectories from the stochastic model are rarely available, requiring the derivation of suitable numerical methods for the synthetic data generation. To obtain summaries that are less sensitive to the intrinsic stochasticity of the model, we propose to build up the statistical method (e.g. the choice of the summary statistics) on the underlying structural properties of the model. Here, we focus on the existence of an invariant measure and we map the data to their estimated invariant density and invariant spectral density. Then, to ensure that these model properties are kept in the synthetic data generation, we adopt measure-preserving numerical splitting schemes. The derived property-based and measure-preserving ABC method is illustrated on the broad class of partially observed Hamiltonian type SDEs, both with simulated data and with real electroencephalography data. The derived summaries are particularly robust to the model simulation, and this fact, combined with the proposed reliable numerical scheme, yields accurate ABC inference. In contrast, the inference returned using standard numerical methods (Euler–Maruyama discretisation) fails. The proposed ingredients can be incorporated into any type of ABC algorithm and directly applied to all SDEs that are characterised by an invariant distribution and for which a measure-preserving numerical method can be derived.

     

Bayesian inference for stable Lévy–driven stochastic differential equations with high‐frequency data

In this paper, we consider parametric Bayesian inference for stochastic differential equations driven by a pure‐jump stable Lévy process, which is observed at high frequency. In most cases of practical interest, the likelihood function is not available; hence, we use a quasi‐likelihood and place an associated prior on the unknown parameters. It is shown under regularity conditions that there is a Bernstein–von Mises theorem associated to the posterior. We then develop a Markov chain Monte Carlo algorithm for Bayesian inference, and assisted with theoretical results, we show how to scale Metropolis–Hastings proposals when the frequency of the data grows, in order to prevent the acceptance ratio from going to zero in the large data limit. Our algorithm is presented on numerical examples that help verify our theoretical findings.     

Spatial modelling of gene frequencies in the presence of undetectable alleles

Bayesian hierarchical models are developed to estimate the frequencies of the alleles at the HLA-C locus in the presence of non-identifiable alleles and possible spatial correlations in a large but sparse, spatially defined database from Papua New Guinea. Bayesian model selection methods are applied to investigate the effects of altitude and language on the genetic diversity of HLA-C alleles. The general model includes fixed altitudinal effects, random language effects and random spatially structured location effects. Conditional autoregressive priors are used to incorporate the geographical structure of the map, and Markov chain Monte Carlo simulation methods are applied for estimation and inference. The results show that HLA-C allele frequencies are explained more by linguistic than altitudinal differences, indicating that genetic diversity at this locus in Papua New Guinea probably tracks population movements and is less influenced by natural selection than is variation at HLA-A and HLA-B.     

A Bayesian approach to mixture cure models with spatial frailties for population‐based cancer relative survival data

As the treatments of cancer progress, a certain number of cancers are curable if diagnosed early. In population‐based cancer survival studies, cure is said to occur when mortality rate of the cancer patients returns to the same level as that expected for the general cancer‐free population. The estimates of cure fraction are of interest to both cancer patients and health policy makers. Mixture cure models have been widely used because the model is easy to interpret by separating the patients into two distinct groups. Usually parametric models are assumed for the latent distribution for the uncured patients. The estimation of cure fraction from the mixture cure model may be sensitive to misspecification of latent distribution. We propose a Bayesian approach to mixture cure model for population‐based cancer survival data, which can be extended to county‐level cancer survival data. Instead of modeling the latent distribution by a fixed parametric distribution, we use a finite mixture of the union of the lognormal, loglogistic, and Weibull distributions. The parameters are estimated using the Markov chain Monte Carlo method. Simulation study shows that the Bayesian method using a finite mixture latent distribution provides robust inference of parameter estimates. The proposed Bayesian method is applied to relative survival data for colon cancer patients from the Surveillance, Epidemiology, and End Results (SEER) Program to estimate the cure fractions. The Canadian Journal of Statistics 40: 40–54; 2012 © 2012 Statistical Society of Canada     

The multinomial logistic regression model for predicting the discharge status after liver transplantation: estimation and diagnostics analysis

The multinomial logistic regression model (MLRM) can be interpreted as a natural extension of the binomial model with logit link function to situations where the response variable can have three or more possible outcomes. In addition, when the categories of the response variable are nominal, the MLRM can be expressed in terms of two or more logistic models and analyzed in both frequentist and Bayesian approaches. However, few discussions about post modeling in categorical data models are found in the literature, and they mainly use Bayesian inference. The objective of this work is to present classic and Bayesian diagnostic measures for categorical data models. These measures are applied to a dataset (status) of patients undergoing kidney transplantation.     

Accelerating inference for diffusions observed with measurement error and large sample sizes using approximate Bayesian computation

In recent years, dynamical modelling has been provided with a range of breakthrough methods to perform exact Bayesian inference. However, it is often computationally unfeasible to apply exact statistical methodologies in the context of large data sets and complex models. This paper considers a nonlinear stochastic differential equation model observed with correlated measurement errors and an application to protein folding modelling. An approximate Bayesian computation (ABC)-MCMC algorithm is suggested to allow inference for model parameters within reasonable time constraints. The ABC algorithm uses simulations of ‘subsamples’ from the assumed data-generating model as well as a so-called ‘early-rejection’ strategy to speed up computations in the ABC-MCMC sampler. Using a considerate amount of subsamples does not seem to degrade the quality of the inferential results for the considered applications. A simulation study is conducted to compare our strategy with exact Bayesian inference, the latter resulting two orders of magnitude slower than ABC-MCMC for the considered set-up. Finally, the ABC algorithm is applied to a large size protein data. The suggested methodology is fairly general and not limited to the exemplified model and data.     

Bayesian Analysis in Regression Models Using Pseudo-Likelihoods

This article deals with the issue of using a suitable pseudo-likelihood, instead of an integrated likelihood, when performing Bayesian inference about a scalar parameter of interest in the presence of nuisance parameters. The proposed approach has the advantages of avoiding the elicitation on the nuisance parameters and the computation of multidimensional integrals. Moreover, it is particularly useful when it is difficult, or even impractical, to write the full likelihood function.

We focus on Bayesian inference about a scalar regression coefficient in various regression models. First, in the context of non-normal regression-scale models, we give a theroetical result showing that there is no loss of information about the parameter of interest when using a posterior distribution derived from a pseudo-likelihood instead of the correct posterior distribution. Second, we present non trivial applications with high-dimensional, or even infinite-dimensional, nuisance parameters in the context of nonlinear normal heteroscedastic regression models, and of models for binary outcomes and count data, accounting also for possibile overdispersion. In all these situtations, we show that non Bayesian methods for eliminating nuisance parameters can be usefully incorporated into a one-parameter Bayesian analysis.     

中国煤电能源链的生命周期碳排放系数计量

 针对我国煤电高污染的行业背景和电力行业低碳化发展的时代要求,文章应用全生命周期分析方法建立了我国煤电能源链的碳排放计量总模型和各环节的子计量模型,进而通过详细计算得出了我国燃煤电厂单位发电量引致的子环节 当量排放及煤电能源链 当量的总排放数据,对比发现了燃煤发电环节为我国煤电能源链温室气体排放的主要环节,最后对各环节的排放结果作出了综合评价与解释。研究对增进了解我国煤电能源链各单元过程温室气体的产生来源和大小,明晰减排调控的重点方向,实现我国电力行业的低碳化发展,都具有一定的理论和现实意义。     

On the dependent competing risks using Marshall–Olkin bivariate Weibull model: Parameter estimation with different methods

Competing risks models are of great importance in reliability and survival analysis. They are often assumed to have independent causes of failure in literature, which may be unreasonable. In this article, dependent causes of failure are considered by using the Marshall–Olkin bivariate Weibull distribution. After deriving some useful results for the model, we use ML, fiducial inference, and Bayesian methods to estimate the unknown model parameters with a parameter transformation. Simulation studies are carried out to assess the performances of the three methods. Compared with the maximum likelihood method, the fiducial and Bayesian methods could provide better parameter estimation.     

A Bayesian mixture model for differential gene expression

Summary.  We propose model-based inference for differential gene expression, using a nonparametric Bayesian probability model for the distribution of gene intensities under various conditions. The probability model is a mixture of normal distributions. The resulting inference is similar to a popular empirical Bayes approach that is used for the same inference problem. The use of fully model-based inference mitigates some of the necessary limitations of the empirical Bayes method. We argue that inference is no more difficult than posterior simulation in traditional nonparametric mixture-of-normal models. The approach proposed is motivated by a microarray experiment that was carried out to identify genes that are differentially expressed between normal tissue and colon cancer tissue samples. Additionally, we carried out a small simulation study to verify the methods proposed. In the motivating case-studies we show how the nonparametric Bayes approach facilitates the evaluation of posterior expected false discovery rates. We also show how inference can proceed even in the absence of a null sample of known non-differentially expressed scores. This highlights the difference from alternative empirical Bayes approaches that are based on plug-in estimates.     

Maximum-Entropy Prior Uncertainty and Correlation of Statistical Economic Data

Empirical estimates of source statistical economic data such as trade flows, greenhouse gas emissions, or employment figures are always subject to uncertainty (stemming from measurement errors or confidentiality) but information concerning that uncertainty is often missing. This article uses concepts from Bayesian inference and the maximum entropy principle to estimate the prior probability distribution, uncertainty, and correlations of source data when such information is not explicitly provided. In the absence of additional information, an isolated datum is described by a truncated Gaussian distribution, and if an uncertainty estimate is missing, its prior equals the best guess. When the sum of a set of disaggregate data is constrained to match an aggregate datum, it is possible to determine the prior correlations among disaggregate data. If aggregate uncertainty is missing, all prior correlations are positive. If aggregate uncertainty is available, prior correlations can be either all positive, all negative, or a mix of both. An empirical example is presented, which reports relative uncertainties and correlation priors for the County Business Patterns database. In this example, relative uncertainties range from 1% to 80% and 20% of data pairs exhibit correlations below ?0.9 or above 0.9. Supplementary materials for this article are available online.     

Nonparametric Bayesian inference in applications

Nonparametric Bayesian (BNP) inference is concerned with inference for infinite dimensional parameters, including unknown distributions, families of distributions, random mean functions and more. Better computational resources and increased use of massive automated or semi-automated data collection makes BNP models more and more common. We briefly review some of the main classes of models, with an emphasis on how they arise from applied research questions, and focus in more depth only on BNP models for spatial inference as a good example of a class of inference problems where BNP models can successfully address limitations of parametric inference.