The conflict between improper priors and robustness

Most methods for assessing the sensitivity of the posterior to the prior do not work if the prior is improper. We characterize the neighborhoods of priors that permit non-trivial sensitivity analysis for improper priors. We show that these neighborhoods are not “tail rich”, i.e. they do not contain measures with a variety of tail behavior. Thus there are no neighborhoods that are simultaneously non-trivial and rich. We also show that using the formal structure of finitely additive priors does not solve the problem. We suggest some directions for addressing the problem. In particular, we consider replacing the improper prior with a sequence of asymptotically well-behaved data-dependent priors.     

Prbposterior expected loss as a scoring rule for prior distributions

A scoring rule for evaluating the usefulness of an assessed prior distribution should reflect the purpose for which the distribution is to be used. In this paper we suppose that sample data is to become available and that the posterior distribution will be used to estimate some quantity under a quadratic loss function. The utility of a prior distribution is consequently determined by its preposterior expected quadratic loss. It is shown that this loss function has properties desirable in a scoring rule and formulae are derived for calculating the scores it gives in some common problems. Many scoring rules give a very poor score to any improper prior distribution but, in contrast, the scoring rule proposed here provides a meaningful measure for comparing the usefulness of assessed prior distributions and non-informative (improper) prior distributions. Results for making this comparison in various situations are also given.     

A prior for the variance in hierarchical models

The choice of prior distributions for the variances can be important and quite difficult in Bayesian hierarchical and variance component models. For situations where little prior information is available, a ‘nonin-formative’ type prior is usually chosen. ‘Noninformative’ priors have been discussed by many authors and used in many contexts. However, care must be taken using these prior distributions as many are improper and thus, can lead to improper posterior distributions. Additionally, in small samples, these priors can be ‘informative’. In this paper, we investigate a proper ‘vague’ prior, the uniform shrinkage prior (Strawder-man 1971; Christiansen & Morris 1997). We discuss its properties and show how posterior distributions for common hierarchical models using this prior lead to proper posterior distributions. We also illustrate the attractive frequentist properties of this prior for a normal hierarchical model including testing and estimation. To conclude, we generalize this prior to the multivariate situation of a covariance matrix.     

Posterior Bayes factor analysis for an exponential regression model

In the exponential regression model, Bayesian inference concerning the non-linear regression parameter has proved extremely difficult. In particular, standard improper diffuse priors for the usual parameters lead to an improper posterior for the non-linear regression parameter. In a recent paper Ye and Berger (1991) applied the reference prior approach of Bernardo (1979) and Berger and Bernardo (1989) yielding a proper informative prior for . This prior depends on the values of the explanatory variable, goes to 0 as goes to 1, and depends on the specification of a hierarchical ordering of importance of the parameters.This paper explains the failure of the uniform prior to give a proper posterior: the reason is the appearance of the determinant of the information matrix in the posterior density for . We apply the posterior Bayes factor approach of Aitkin (1991) to this problem; in this approach we integrate out nuisance parameters with respect to their conditional posterior density given the parameter of interest. The resulting integrated likelihood for requires only the standard diffuse prior for all the parameters, and is unaffected by orderings of importance of the parameters. Computation of the likelihood for is extremely simple. The approach is applied to the three examples discussed by Berger and Ye and the likelihoods compared with their posterior densities.     

Sensitivity of the fractional Bayes factor to prior distributions

The authors derive a measure of the sensitivity of the fractional Bayes factor, an index which is used to compare models when the priors for their respective parameters are improper, or when there is concern about robustness of the prior specification. They prove that in a large class of problems, this measure is a decreasing function of the fraction of the sample used to update the prior distribution before the models are compared.     

Propriety of posteriors in structured additive regression models: Theory and empirical evidence

Structured additive regression comprises many semiparametric regression models such as generalized additive (mixed) models, geoadditive models, and hazard regression models within a unified framework. In a Bayesian formulation, non-parametric functions, spatial effects and further model components are specified in terms of multivariate Gaussian priors for high-dimensional vectors of regression coefficients. For several model terms, such as penalized splines or Markov random fields, these Gaussian prior distributions involve rank-deficient precision matrices, yielding partially improper priors. Moreover, hyperpriors for the variances (corresponding to inverse smoothing parameters) may also be specified as improper, e.g. corresponding to Jeffreys prior or a flat prior for the standard deviation. Hence, propriety of the joint posterior is a crucial issue for full Bayesian inference in particular if based on Markov chain Monte Carlo simulations. We establish theoretical results providing sufficient (and sometimes necessary) conditions for propriety and provide empirical evidence through several accompanying simulation studies.     

Asymptotic inference for mixture models by using data-dependent priors

For certain mixture models, improper priors are undesirable because they yield improper posteriors. However, proper priors may be undesirable because they require subjective input. We propose the use of specially chosen data-dependent priors. We show that, in some cases, data-dependent priors are the only priors that produce intervals with second-order correct frequentist coverage. The resulting posterior also has another interpretation: it is the product of a fixed prior and a pseudolikelihood.     

Bayesian analysis of elapsed times in continuous‐time Markov chains

The authors consider Bayesian analysis for continuous‐time Markov chain models based on a conditional reference prior. For such models, inference of the elapsed time between chain observations depends heavily on the rate of decay of the prior as the elapsed time increases. Moreover, improper priors on the elapsed time may lead to improper posterior distributions. In addition, an infinitesimal rate matrix also characterizes this class of models. Experts often have good prior knowledge about the parameters of this matrix. The authors show that the use of a proper prior for the rate matrix parameters together with the conditional reference prior for the elapsed time yields a proper posterior distribution. The authors also demonstrate that, when compared to analyses based on priors previously proposed in the literature, a Bayesian analysis on the elapsed time based on the conditional reference prior possesses better frequentist properties. The type of prior thus represents a better default prior choice for estimation software.     

Bayesian Estimation of the Parameters of Bivariate Exponential Distributions

In this article, we develop an empirical Bayesian approach for the Bayesian estimation of parameters in four bivariate exponential (BVE) distributions. We have opted for gamma distribution as a prior for the parameters of the model in which the hyper parameters have been estimated based on the method of moments and maximum likelihood estimates (MLEs). A simulation study was conducted to compute empirical Bayesian estimates of the parameters and their standard errors. We use moment estimators or MLEs to estimate the hyper parameters of the prior distributions. Furthermore, we compare the posterior mode of parameters obtained by different prior distributions and the Bayesian estimates based on gamma priors are very close to the true values as compared to improper priors. We use MCMC method to obtain the posterior mean and compared the same using the improper priors and the classical estimates, MLEs.     

On selecting the best treatment in a generalized linear model

The problem of selecting the best treatment is studied under generalized linear models. For certain balanced designs, it is shown that simple rules are Bayes with respect to any non-informative prior on the treatment effects under any monotone invariant loss. When the nuisance parameters such as block effects are assumed to follow a uniform (improper) prior or a normal prior, Bayes rules are obtained for the normal linear model under more suitable balanced designs, keeping the generality of the loss and the generality of the non-informativeness on the prior of the treatment effects. These results are extended to certain types of informative priors on the treatment effects. When the designs are unbalanced, algorithms based on the Gibbs sampler and the Laplace method are provided to compute the Bayes rules.     

A 3-parameter Gompertz distribution for survival data with competing risks,with an application to breast cancer data

The cumulative incidence function is of great importance in the analysis of survival data when competing risks are present. Parametric modeling of such functions, which are by nature improper, suggests the use of improper distributions. One frequently used improper distribution is that of Gompertz, which captures only monotone hazard shapes. In some applications, however, subdistribution hazard estimates have been observed with unimodal shapes. An extension to the Gompertz distribution is presented which can capture unimodal as well as monotone hazard shapes. Important properties of the proposed distribution are discussed, and the proposed distribution is used to analyze survival data from a breast cancer clinical trial.     

Equality and inequality constrained multivariate linear models: Objective model selection using constrained posterior priors

In objective Bayesian model selection, a well-known problem is that standard non-informative prior distributions cannot be used to obtain a sensible outcome of the Bayes factor because these priors are improper. The use of a small part of the data, i.e., a training sample, to obtain a proper posterior prior distribution has become a popular method to resolve this issue and seems to result in reasonable outcomes of default Bayes factors, such as the intrinsic Bayes factor or a Bayes factor based on the empirical expected-posterior prior.     

On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion)

New methodology for fully Bayesian mixture analysis is developed, making use of reversible jump Markov chain Monte Carlo methods that are capable of jumping between the parameter subspaces corresponding to different numbers of components in the mixture. A sample from the full joint distribution of all unknown variables is thereby generated, and this can be used as a basis for a thorough presentation of many aspects of the posterior distribution. The methodology is applied here to the analysis of univariate normal mixtures, using a hierarchical prior model that offers an approach to dealing with weak prior information while avoiding the mathematical pitfalls of using improper priors in the mixture context.     

A Bayesian estimation of lag lengths in distributed lag models

Dynamic regression models are widely used because they express and model the behaviour of a system over time. In this article, two dynamic regression models, the distributed lag (DL) model and the autoregressive distributed lag model, are evaluated focusing on their lag lengths. From a classical statistics point of view, there are various methods to determine the number of lags, but none of them are the best in all situations. This is a serious issue since wrong choices will provide bad estimates for the effects of the regressors on the response variable. We present an alternative for the aforementioned problems by considering a Bayesian approach. The posterior distributions of the numbers of lags are derived under an improper prior for the model parameters. The fractional Bayes factor technique [A. O'Hagan, Fractional Bayes factors for model comparison (with discussion), J. R. Statist. Soc. B 57 (1995), pp. 99–138] is used to handle the indeterminacy in the likelihood function caused by the improper prior. The zero-one loss function is used to penalize wrong decisions. A naive method using the specified maximum number of DLs is also presented. The proposed and the naive methods are verified using simulation data. The results are promising for the method we proposed. An illustrative example with a real data set is provided.     

Calibrated Bayes factors for model comparison

A Bayes factor between two models can be greatly affected by the prior distributions on the model parameters. When prior information is weak, very dispersed proper prior distributions are known to create a problem for the Bayes factor when competing models differ in dimension, and it is of even greater concern when one of the models is of infinite dimension. Therefore, we propose an innovative method which uses training samples to calibrate the prior distributions so that they achieve a reasonable level of ‘information’. Then the calibrated Bayes factor can be computed over the remaining data. This method makes no assumption on model forms (parametric or nonparametric) and can be used with both proper and improper priors. We illustrate, through simulation studies and a real data example, that the calibrated Bayes factor yields robust and reliable model preferences under various situations.     

A Comparison of Bayes Factors for Separated Models: Some Simulation Results

Some alternative Bayes Factors: Intrinsic, Posterior, and Fractional have been proposed to overcome the difficulties presented when prior information is weak and improper prior are used. Additional difficulties also appear when the models are separated or non nested. This article presents both simulation results and some illustrative examples analysis comparing these alternative Bayes factors to discriminate among the Lognormal, the Weibull, the Gamma, and the Exponential distributions. Simulation results are obtained for different sample sizes generated from the distributions. Results from simulations indicates that these alternative Bayes factors are useful for comparing non nested models. The simulations also show some similar behavior and that when both models are true they choose the simplest model. Some illustrative example are also presented.     

Admissibility in quadratically regular problems and recurrence of symmetric Markov chains: Why the connection?

In the expository paper, a sufficient condition is discussed for the almost admissibility of formal Bayes rules in quadratically regular problems. This sufficient condition is equivalent to a recurrence property of a natural symmetric Markov chain constructed from the model and the improper prior. Some simple examples involving translation parameter models illustrate the results.     

The formal posterior of a standard flat prior in MANOVA is incoherent

Summary A standard improper prior for the parameters of a MANOVA model is shown to yield an inference that is incoherent in the sense of Heath and Sudderth. The proof of incoherence is based on the fact that the formal Bayes estimate, sayδ ₀ , of the covariance matrix based on the improper prior and a certain bounded loss function is uniformly inadmissible in that there is another estimatorδ _l and an ɛ>0 such that the risk functions satisfyR(δ _l ,Σ)⩽R δ ₀ ,Σ)−ε for all values of the covariance matrix Σ. The estimatorδ _I is formal Bayes for an alternative improper prior which leads to a coherent inference. Research supported by National Science Foundation grants DMS-89-22607 (for Eaton) and DMS-9123358 (for Sudderth).     

Predictive Inference from a Two-Parameter Rayleigh Life Model Given a Doubly Censored Sample

This article is concerned with making predictive inference on the basis of a doubly censored sample from a two-parameter Rayleigh life model. We derive the predictive distributions for a single future response, the ith future response, and several future responses. We use the Bayesian approach in conjunction with an improper flat prior for the location parameter and an independent proper conjugate prior for the scale parameter to derive the predictive distributions. We conclude with a numerical example in which the effect of the hyperparameters on the mean and standard deviation of the predictive density is assessed.     

Improper priors for translation families

We consider the problem of inference when sampling from a translation family with an improper prior. Properties of the formal Bayes inference will be studied. We give conditions (on the prior and/or the family) guaranteeing the HS-coherence (see Heath and Sudderth, Ann. Statist. 6 (1978), 333–345) of the formal Bayes posterior. Since HS-coherence is equivalent to being a posterior of a finitely additive prior, all coherence results imply the existence of a finitely additive prior which has the formal Bayes inference as a posterior.