Propriety of posteriors in structured additive regression models: Theory and empirical evidence

Structured additive regression comprises many semiparametric regression models such as generalized additive (mixed) models, geoadditive models, and hazard regression models within a unified framework. In a Bayesian formulation, non-parametric functions, spatial effects and further model components are specified in terms of multivariate Gaussian priors for high-dimensional vectors of regression coefficients. For several model terms, such as penalized splines or Markov random fields, these Gaussian prior distributions involve rank-deficient precision matrices, yielding partially improper priors. Moreover, hyperpriors for the variances (corresponding to inverse smoothing parameters) may also be specified as improper, e.g. corresponding to Jeffreys prior or a flat prior for the standard deviation. Hence, propriety of the joint posterior is a crucial issue for full Bayesian inference in particular if based on Markov chain Monte Carlo simulations. We establish theoretical results providing sufficient (and sometimes necessary) conditions for propriety and provide empirical evidence through several accompanying simulation studies.     

A class of neutral to the right priors induced by superposition of beta processes

A random distribution function on the positive real line which belongs to the class of neutral to the right priors is defined. It corresponds to the superposition of independent beta processes at the cumulative hazard level. The definition is constructive and starts with a discrete time process with random probability masses obtained from suitably defined products of independent beta random variables. The continuous time version is derived as the corresponding infinitesimal weak limit and is described in terms of completely random measures. It takes the interpretation of the survival distribution resulting from independent competing failure times. We discuss prior specification and illustrate posterior inference on a real data example.     

Efficient network meta-analysis: A confidence distribution approach

Network meta-analysis synthesizes several studies of multiple treatment comparisons to simultaneously provide inference for all treatments in the network. It can often strengthen inference on pairwise comparisons by borrowing evidence from other comparisons in the network. Current network meta-analysis approaches are derived from either conventional pairwise meta-analysis or hierarchical Bayesian methods. This paper introduces a new approach for network meta-analysis by combining confidence distributions (CDs). Instead of combining point estimators from individual studies in the conventional approach, the new approach combines CDs, which contain richer information than point estimators, and thus achieves greater efficiency in its inference. The proposed CD approach can efficiently integrate all studies in the network and provide inference for all treatments, even when individual studies contain only comparisons of subsets of the treatments. Through numerical studies with real and simulated data sets, the proposed approach is shown to outperform or at least equal the traditional pairwise meta-analysis and a commonly used Bayesian hierarchical model. Although the Bayesian approach may yield comparable results with a suitably chosen prior, it is highly sensitive to the choice of priors (especially for the between-trial covariance structure), which is often subjective. The CD approach is a general frequentist approach and is prior-free. Moreover, it can always provide a proper inference for all the treatment effects regardless of the between-trial covariance structure.     

On conjugate families and Jeffreys priors for von Mises–Fisher distributions

This paper discusses characteristics of standard conjugate priors and their induced posteriors in Bayesian inference for von Mises–Fisher distributions, using either the canonical natural exponential family or the more commonly employed polar coordinate parameterizations. We analyze when standard conjugate priors as well as posteriors are proper, and investigate the Jeffreys prior for the von Mises–Fisher family. Finally, we characterize the proper distributions in the standard conjugate family of the (matrix-valued) von Mises–Fisher distributions on Stiefel manifolds.     

Non parametric mixture priors based on an exponential random scheme

We propose a general procedure for constructing nonparametric priors for Bayesian inference. Under very general assumptions, the proposed prior selects absolutely continuous distribution functions, hence it can be useful with continuous data. We use the notion ofFeller-type approximation, with a random scheme based on the natural exponential family, in order to construct a large class of distribution functions. We show how one can assign a probability to such a class and discuss the main properties of the proposed prior, namedFeller prior. Feller priors are related to mixture models with unknown number of components or, more generally, to mixtures with unknown weight distribution. Two illustrations relative to the estimation of a density and of a mixing distribution are carried out with respect to well known data-set in order to evaluate the performance of our procedure. Computations are performed using a modified version of an MCMC algorithm which is briefly described.     

Improved estimation procedures for multilevel models with binary response: a case-study

During recent years, analysts have been relying on approximate methods of inference to estimate multilevel models for binary or count data. In an earlier study of random-intercept models for binary outcomes we used simulated data to demonstrate that one such approximation, known as marginal quasi-likelihood, leads to a substantial attenuation bias in the estimates of both fixed and random effects whenever the random effects are non-trivial. In this paper, we fit three-level random-intercept models to actual data for two binary outcomes, to assess whether refined approximation procedures, namely penalized quasi-likelihood and second-order improvements to marginal and penalized quasi-likelihood, also underestimate the underlying parameters. The extent of the bias is assessed by two standards of comparison: exact maximum likelihood estimates, based on a Gauss–Hermite numerical quadrature procedure, and a set of Bayesian estimates, obtained from Gibbs sampling with diffuse priors. We also examine the effectiveness of a parametric bootstrap procedure for reducing the bias. The results indicate that second-order penalized quasi-likelihood estimates provide a considerable improvement over the other approximations, but all the methods of approximate inference result in a substantial underestimation of the fixed and random effects when the random effects are sizable. We also find that the parametric bootstrap method can eliminate the bias but is computationally very intensive.     

Slice sampling mixture models

We propose a more efficient version of the slice sampler for Dirichlet process mixture models described by Walker (Commun. Stat., Simul. Comput. 36:45–54, 2007). This new sampler allows for the fitting of infinite mixture models with a wide-range of prior specifications. To illustrate this flexibility we consider priors defined through infinite sequences of independent positive random variables. Two applications are considered: density estimation using mixture models and hazard function estimation. In each case we show how the slice efficient sampler can be applied to make inference in the models. In the mixture case, two submodels are studied in detail. The first one assumes that the positive random variables are Gamma distributed and the second assumes that they are inverse-Gaussian distributed. Both priors have two hyperparameters and we consider their effect on the prior distribution of the number of occupied clusters in a sample. Extensive computational comparisons with alternative “conditional” simulation techniques for mixture models using the standard Dirichlet process prior and our new priors are made. The properties of the new priors are illustrated on a density estimation problem.     

Bayesian revision of a prior given prior-data conflict,expert opinion,or a similar insight: a large-deviation approach

Learning from model diagnostics that a prior distribution must be replaced by one that conflicts less with the data raises the question of which prior should instead be used for inference and decision. The same problem arises when a decision maker learns that one or more reliable experts express unexpected beliefs. In both cases, coherence of the solution would be guaranteed by applying Bayes's theorem to a distribution of prior distributions that effectively assigns the initial prior distribution a probability arbitrarily close to 1. The new distribution for inference would then be the distribution of priors conditional on the insight that the prior distribution lies in a closed convex set that does not contain the initial prior. A readily available distribution of priors needed for such conditioning is the law of the empirical distribution of sufficiently large number of independent parameter values drawn from the initial prior. According to the Gibbs conditioning principle from the theory of large deviations, the resulting new prior distribution minimizes the entropy relative to the initial prior. While minimizing relative entropy accommodates the necessity of going beyond the initial prior without departing from it any more than the insight demands, the large-deviation derivation also ensures the advantages of Bayesian coherence. This approach is generalized to uncertain insights by allowing the closed convex set of priors to be random.     

Geodesic Lagrangian Monte Carlo over the space of positive definite matrices: with application to Bayesian spectral density estimation

We present geodesic Lagrangian Monte Carlo, an extension of Hamiltonian Monte Carlo for sampling from posterior distributions defined on general Riemannian manifolds. We apply this new algorithm to Bayesian inference on symmetric or Hermitian positive definite (PD) matrices. To do so, we exploit the Riemannian structure induced by Cartan's canonical metric. The geodesics that correspond to this metric are available in closed-form and – within the context of Lagrangian Monte Carlo – provide a principled way to travel around the space of PD matrices. Our method improves Bayesian inference on such matrices by allowing for a broad range of priors, so we are not limited to conjugate priors only. In the context of spectral density estimation, we use the (non-conjugate) complex reference prior as an example modelling option made available by the algorithm. Results based on simulated and real-world multivariate time series are presented in this context, and future directions are outlined.     

Bayesian semiparametric modeling and inference with mixtures of?symmetric distributions

We propose a semiparametric modeling approach for mixtures of symmetric distributions. The mixture model is built from a common symmetric density with different components arising through different location parameters. This structure ensures identifiability for mixture components, which is a key feature of the model as it allows applications to settings where primary interest is inference for the subpopulations comprising the mixture. We focus on the two-component mixture setting and develop a Bayesian model using parametric priors for the location parameters and for the mixture proportion, and a nonparametric prior probability model, based on Dirichlet process mixtures, for the random symmetric density. We present an approach to inference using Markov chain Monte Carlo posterior simulation. The performance of the model is studied with a simulation experiment and through analysis of a rainfall precipitation data set as well as with data on eruptions of the Old Faithful geyser.     

Non-informative priors for the common mean in the one-way random effects model with heterogeneous error variances

The paper develops some objective priors for the common mean in the one-way random effects model with heterogeneous error variances. We derive the first and second order matching priors and reference priors. It turns out that the second order matching prior matches the alternative coverage probabilities up to the second order, and is also an HPD matching prior. However, derived reference priors just satisfy a first order matching criterion. Our simulation studies indicate that the second order matching prior performs better than the reference prior and the Jeffreys prior in terms of matching the target coverage probabilities in a frequentist sense. We also illustrate our results using real data.     

Bayesian approach to estimation of intraclass correlation using reference prior

For the balanced variance component model when the intraclass correlation coefficient is of interest, Bayesian analysis is often appropriate. Berger and Bernardo’s (1992a) grouped ordering reference prior approach is used to analyze this model. The reference priors are developed and compared for the posterior inference with real and simulated data. We examine whether the reference priors satisfy the probability-matching criterion. Further, the reference prior is shown to be good in the sense of correct frequentist coverage probability of the posterior quantile.     

Bayesian Recovery of the Initial Condition for the Heat Equation

We study a Bayesian approach to recovering the initial condition for the heat equation from noisy observations of the solution at a later time. We consider a class of prior distributions indexed by a parameter quantifying “smoothness” and show that the corresponding posterior distributions contract around the true parameter at a rate that depends on the smoothness of the true initial condition and the smoothness and scale of the prior. Correct combinations of these characteristics lead to the optimal minimax rate. One type of priors leads to a rate-adaptive Bayesian procedure. The frequentist coverage of credible sets is shown to depend on the combination of the prior and true parameter as well, with smoother priors leading to zero coverage and rougher priors to (extremely) conservative results. In the latter case, credible sets are much larger than frequentist confidence sets, in that the ratio of diameters diverges to infinity. The results are numerically illustrated by a simulated data example.     

Random variate generation from D-distributions

Within the context of non-parametric Bayesian inference, Dykstra and Laud (1981) define an extended gamma (EG) process and use it as a prior on increasing hazard rates. The attractive features of the extended gamma (EG) process, among them its capability to index distribution functions that are absolutely continuous, are offset by the intractable nature of the computation that needs to be performed. Sampling based approaches such as the Gibbs Sampler can alleviate these difficulties but the EG processes then give rise to the problem of efficient random variate generation from a class of distributions called D-distributions. In this paper, we describe a novel technique for sampling from such distributions, thereby providing an efficient computation procedure for non-parametric Bayesian inference with a rich class of priors for hazard rates.     

Bayesian Methods for Missing Covariates in Cure Rate Models

We propose methods for Bayesian inference for missing covariate data with a novel class of semi-parametric survival models with a cure fraction. We allow the missing covariates to be either categorical or continuous and specify a parametric distribution for the covariates that is written as a sequence of one dimensional conditional distributions. We assume that the missing covariates are missing at random (MAR) throughout. We propose an informative class of joint prior distributions for the regression coefficients and the parameters arising from the covariate distributions. The proposed class of priors are shown to be useful in recovering information on the missing covariates especially in situations where the missing data fraction is large. Properties of the proposed prior and resulting posterior distributions are examined. Also, model checking techniques are proposed for sensitivity analyses and for checking the goodness of fit of a particular model. Specifically, we extend the Conditional Predictive Ordinate (CPO) statistic to assess goodness of fit in the presence of missing covariate data. Computational techniques using the Gibbs sampler are implemented. A real data set involving a melanoma cancer clinical trial is examined to demonstrate the methodology.     

A matching prior based on the modified profile likelihood in a generalized Weibull stress‐strength model

Bayesian inference of a generalized Weibull stress‐strength model (SSM) with more than one strength component is considered. For this problem, properly assigning priors for the reliabilities is challenging due to the presence of nuisance parameters. Matching priors, which are priors matching the posterior probabilities of certain regions with their frequentist coverage probabilities, are commonly used but difficult to derive in this problem. Instead, we apply an alternative method and derive a matching prior based on a modification of the profile likelihood. Simulation studies show that this proposed prior performs well in terms of frequentist coverage and estimation even when the sample sizes are minimal. The prior is applied to two real datasets. The Canadian Journal of Statistics 41: 83–97; 2013 © 2012 Statistical Society of Canada     

Posterior consistency in frailty models and simulation studies to test the presence of random effects

The estimation of random effects in frailty models is an important problem in survival analysis. Testing for the presence of random effects can be essential to improving model efficiency. Posterior consistency in dispersion parameters and coefficients of the frailty model was demonstrated in theory and simulations using the posterior induced by Cox’s partial likelihood and simple priors. We also conducted simulation studies to test for the presence of random effects; the proposed method performed well in several simulations. Data analysis was also conducted. The proposed method is easily tractable and can be used to develop various methods for Bayesian inference in frailty models.     

Prior specification of neighbourhood and interaction structure in binary Markov random fields

We formulate a prior distribution for the energy function of stationary binary Markov random fields (MRFs) defined on a rectangular lattice. In the prior we assign distributions to all parts of the energy function. In particular we define priors for the neighbourhood structure of the MRF, what interactions to include in the model, and for potential values. We define a reversible jump Markov chain Monte Carlo (RJMCMC) procedure to simulate from the corresponding posterior distribution when conditioned to an observed scene. Thereby we are able to learn both the neighbourhood structure and the parametric form of the MRF from the observed scene. We circumvent evaluations of the intractable normalising constant of the MRF when running the RJMCMC algorithm by adopting a previously defined approximate auxiliary variable algorithm. We demonstrate the usefulness of our prior in two simulation examples and one real data example.     

Exponential-Bound Property of Estimators and Variable Selection in Generalized Additive Models

In this article, utilizing a scale mixture of skew-normal distribution in which mixing random variable is assumed to follow a mixture model with varying weights for each observation, we introduce a generalization of skew-normal linear regression model with the aim to provide resistant results. This model, which also includes the skew-slash distribution in a particular case, allows us to accommodate and detect outlying observations under the skew-normal linear regression model. Inferences about the model are carried out through the empirical Bayes approach. The conditions for propriety of the posterior and for existence of posterior moments are given under the standard noninformative priors for regression and scale parameters as well as proper prior for skewness parameter. Then, for Bayesian inference, a Markov chain Monte Carlo method is described. Since posterior results depend on the prior hyperparameters, we estimate them adopting the empirical Bayes method as well as using a Monte Carlo EM algorithm. Furthermore, to identify possible outliers, we also apply the Bayes factor obtained through the generalized Savage-Dickey density ratio. Examining the proposed approach on simulated instance and real data, it is found to provide not only satisfactory parameter estimates rather allow identifying outliers favorably.     

Noninformative priors for the between-group variance in the unbalanced one-way random effects model with heterogeneous error variances

For the unbalanced one-way random effects model with heterogeneous error variances, we propose the non-informative priors for the between-group variance and develop the first- and second-order matching priors. It turns out that the second-order matching priors do not exist and the reference prior and Jeffreys prior do not satisfy a first-order matching criterion. We also show that the first-order matching prior meets the frequentist target coverage probabilities much better than the Jeffreys prior and reference prior through simulation study, and the Bayesian credible intervals based on the matching prior and reference prior give shorter intervals than the existing confidence intervals by examples.