On predictive distributions and Bayesian networks

In this paper we are interested in discrete prediction problems for a decision-theoretic setting, where the task is to compute the predictive distribution for a finite set of possible alternatives. This question is first addressed in a general Bayesian framework, where we consider a set of probability distributions defined by some parametric model class. Given a prior distribution on the model parameters and a set of sample data, one possible approach for determining a predictive distribution is to fix the parameters to the instantiation with the maximum a posteriori probability. A more accurate predictive distribution can be obtained by computing the evidence (marginal likelihood), i.e., the integral over all the individual parameter instantiations. As an alternative to these two approaches, we demonstrate how to use Rissanen's new definition of stochastic complexity for determining predictive distributions, and show how the evidence predictive distribution with Jeffrey's prior approaches the new stochastic complexity predictive distribution in the limit with increasing amount of sample data. To compare the alternative approaches in practice, each of the predictive distributions discussed is instantiated in the Bayesian network model family case. In particular, to determine Jeffrey's prior for this model family, we show how to compute the (expected) Fisher information matrix for a fixed but arbitrary Bayesian network structure. In the empirical part of the paper the predictive distributions are compared by using the simple tree-structured Naive Bayes model, which is used in the experiments for computational reasons. The experimentation with several public domain classification datasets suggest that the evidence approach produces the most accurate predictions in the log-score sense. The evidence-based methods are also quite robust in the sense that they predict surprisingly well even when only a small fraction of the full training set is used.     

Bayesian sample size determination for phase IIA clinical trials using historical data and semi‐parametric prior's elicitation

The Simon's two‐stage design is the most commonly applied among multi‐stage designs in phase IIA clinical trials. It combines the sample sizes at the two stages in order to minimize either the expected or the maximum sample size. When the uncertainty about pre‐trial beliefs on the expected or desired response rate is high, a Bayesian alternative should be considered since it allows to deal with the entire distribution of the parameter of interest in a more natural way. In this setting, a crucial issue is how to construct a distribution from the available summaries to use as a clinical prior in a Bayesian design. In this work, we explore the Bayesian counterparts of the Simon's two‐stage design based on the predictive version of the single threshold design. This design requires specifying two prior distributions: the analysis prior, which is used to compute the posterior probabilities, and the design prior, which is employed to obtain the prior predictive distribution. While the usual approach is to build beta priors for carrying out a conjugate analysis, we derived both the analysis and the design distributions through linear combinations of B‐splines. The motivating example is the planning of the phase IIA two‐stage trial on anti‐HER2 DNA vaccine in breast cancer, where initial beliefs formed from elicited experts' opinions and historical data showed a high level of uncertainty. In a sample size determination problem, the impact of different priors is evaluated.     

Bayesian Conditional Mean Estimation in Log‐Normal Linear Regression Models with Finite Quadratic Expected Loss

Log‐normal linear regression models are popular in many fields of research. Bayesian estimation of the conditional mean of the dependent variable is problematic as many choices of the prior for the variance (on the log‐scale) lead to posterior distributions with no finite moments. We propose a generalized inverse Gaussian prior for this variance and derive the conditions on the prior parameters that yield posterior distributions of the conditional mean of the dependent variable with finite moments up to a pre‐specified order. The conditions depend on one of the three parameters of the suggested prior; the other two have an influence on inferences for small and medium sample sizes. A second goal of this paper is to discuss how to choose these parameters according to different criteria including the optimization of frequentist properties of posterior means.     

Bayesian analysis of a multiple-recapture model

In the Bayesian analysis of a multiple-recapture census, different diffuse prior distributions can lead to markedly different inferences about the population size N. Through consideration of the Fisher information matrix it is shown that the number of captures in each sample typically provides little information about N. This suggests that if there is no prior information about capture probabilities, then knowledge of just the sample sizes and not the number of recaptures should leave the distribution of Nunchanged. A prior model that has this property is identified and the posterior distribution is examined. In particular, asymptotic estimates of the posterior mean and variance are derived. Differences between Bayesian and classical point and interval estimators are illustrated through examples.     

On the choice of the prior distribution in hypergeometric sampling

Information in a statistical procedure arising from sources other than sampling is called prior information, and its incorporation into the procedure forms the basis of the Bayesian approach to statistics. Under hypergeometric sampling, methodology is developed which quantifies the amount of information provided by the sample data relative to that provided by the prior distribution and allows for a ranking of prior distributions with respect to conservativeness, where conservatism refers to restraint of extraneous information embedded in any prior distribution. The most conservative prior distribution from a specified class (each member of which carries the available prior information) is that prior distribution within the class over which the likelihood function has the greatest average domination. Four different families of prior distributions are developed by considering a Bayesian approach to the formation of lots. The most conservative prior distribution from each of the four families of prior distributions is determined and compared for the situation when no prior information is available. The results of the comparison advocate the use of the Polya (beta-binomial) prior distribution in hypergeometric sampling.     

A prior for the variance in hierarchical models

The choice of prior distributions for the variances can be important and quite difficult in Bayesian hierarchical and variance component models. For situations where little prior information is available, a ‘nonin-formative’ type prior is usually chosen. ‘Noninformative’ priors have been discussed by many authors and used in many contexts. However, care must be taken using these prior distributions as many are improper and thus, can lead to improper posterior distributions. Additionally, in small samples, these priors can be ‘informative’. In this paper, we investigate a proper ‘vague’ prior, the uniform shrinkage prior (Strawder-man 1971; Christiansen & Morris 1997). We discuss its properties and show how posterior distributions for common hierarchical models using this prior lead to proper posterior distributions. We also illustrate the attractive frequentist properties of this prior for a normal hierarchical model including testing and estimation. To conclude, we generalize this prior to the multivariate situation of a covariance matrix.     

Fiducial and Confidence Distributions for Real Exponential Families

We develop an easy and direct way to define and compute the fiducial distribution of a real parameter for both continuous and discrete exponential families. Furthermore, such a distribution satisfies the requirements to be considered a confidence distribution. Many examples are provided for models, which, although very simple, are widely used in applications. A characterization of the families for which the fiducial distribution coincides with a Bayesian posterior is given, and the strict connection with Jeffreys prior is shown. Asymptotic expansions of fiducial distributions are obtained without any further assumptions, and again, the relationship with the objective Bayesian analysis is pointed out. Finally, using the Edgeworth expansions, we compare the coverage of the fiducial intervals with that of other common intervals, proving the good behaviour of the former.     

Predictive control of posterior robustness for sample size choice in a Bernoulli model

In this article we consider the sample size determination problem in the context of robust Bayesian parameter estimation of the Bernoulli model. Following a robust approach, we consider classes of conjugate Beta prior distributions for the unknown parameter. We assume that inference is robust if posterior quantities of interest (such as point estimates and limits of credible intervals) do not change too much as the prior varies in the selected classes of priors. For the sample size problem, we consider criteria based on predictive distributions of lower bound, upper bound and range of the posterior quantity of interest. The sample size is selected so that, before observing the data, one is confident to observe a small value for the posterior range and, depending on design goals, a large (small) value of the lower (upper) bound of the quantity of interest. We also discuss relationships with and comparison to non robust and non informative Bayesian methods.     

A BAYESIAN METHOD FOR THE CHOICE OF THE SAMPLE SIZE IN EQUIVALENCE TRIALS

In this paper we consider a Bayesian predictive approach to sample size determination in equivalence trials. Equivalence experiments are conducted to show that the unknown difference between two parameters is small. For instance, in clinical practice this kind of experiment aims to determine whether the effects of two medical interventions are therapeutically similar. We declare an experiment successful if an interval estimate of the effects‐difference is included in a set of values of the parameter of interest indicating a negligible difference between treatment effects (equivalence interval). We derive two alternative criteria for the selection of the optimal sample size, one based on the predictive expectation of the interval limits and the other based on the predictive probability that these limits fall in the equivalence interval. Moreover, for both criteria we derive a robust version with respect to the choice of the prior distribution. Numerical results are provided and an application is illustrated when the normal model with conjugate prior distributions is assumed.     

Confidence sets as programming problems

Optimizing criteria for choosing a confidence set for a parameter are formulated as mathematical programming problems. The two optimizing criteria, probability of coverage and size of set, give rise to a pair of inverse programming problems. Several examples are worked out. The programming problems are then formulated to allow the incorporation of partial information about the parameter. By varying the family of prior distributions, a continuum of problems from the frequency approach to a Bayesian approach is obtained. Some examples are considered in which the family of priors contains more than one but not all prior distributions.     

Sensitivity of a bayesian inference to prior assumptions

The sensitivity of-a Bayesian inference to prior assumptions is examined by Monte Carlo simulation for the beta-binomial conjugate family of distributions. Results for the effect on a Bayesian probability interval of the binomial parameter indicate that the Bayesian inference is for the most part quite sensitive to misspecification of the prior distribution. The magnitude of the sensitivity depends primarily on the difference of assigned means and variances from the respective means and variances of the actually-sampled prior distributions. The effect of a disparity in form between the assigned prior and actually-sampled distributions was less important for the cases tested.     

An information-theoretic approach to incorporating prior information in binomial sampling

The incorporation of prior information about θ, where θ is the success probability in a binomial sampling model, is an essential feature of Bayesian statistics. Methodology based on information-theoretic concepts is introduced which (a) quantifies the amount of information provided by the sample data relative to that provided by the prior distribution and (b) allows for a ranking of prior distributions with respect to conservativeness, where conservatism refers to restraint of extraneous information about θ which is embedded in any prior distribution. In effect, the most conservative prior distribution from a specified class (each member o f which carries the available prior information about θ) is that prior distribution within the class over which the likelihood function has the greatest average domination. The most conservative prior distributions from five different families of prior distributions over the interval (0,1) including the beta distribution are determined and compared for three situations: (1) no prior estimate of θ is available, (2) a prior point estimate or θ is available, and (3) a prior interval estimate of θ is available. The results of the comparisons not only advocate the use of the beta prior distribution in binomial sampling but also indicate which particular one to use in the three aforementioned situations.     

A Bayesian approach to non-parametric monotone function estimation

Summary.  The paper proposes two Bayesian approaches to non-parametric monotone function estimation. The first approach uses a hierarchical Bayes framework and a characterization of smooth monotone functions given by Ramsay that allows unconstrained estimation. The second approach uses a Bayesian regression spline model of Smith and Kohn with a mixture distribution of constrained normal distributions as the prior for the regression coefficients to ensure the monotonicity of the resulting function estimate. The small sample properties of the two function estimators across a range of functions are provided via simulation and compared with existing methods. Asymptotic results are also given that show that Bayesian methods provide consistent function estimators for a large class of smooth functions. An example is provided involving economic demand functions that illustrates the application of the constrained regression spline estimator in the context of a multiple-regression model where two functions are constrained to be monotone.     

Bayesian–frequentist hybrid model with application to the analysis of gene copy number changes

Gene copy number (GCN) changes are common characteristics of many genetic diseases. Comparative genomic hybridization (CGH) is a new technology widely used today to screen the GCN changes in mutant cells with high resolution genome-wide. Statistical methods for analyzing such CGH data have been evolving. Existing methods are either frequentist's or full Bayesian. The former often has computational advantage, while the latter can incorporate prior information into the model, but could be misleading when one does not have sound prior information. In an attempt to take full advantages of both approaches, we develop a Bayesian-frequentist hybrid approach, in which a subset of the model parameters is inferred by the Bayesian method, while the rest parameters by the frequentist's. This new hybrid approach provides advantages over those of the Bayesian or frequentist's method used alone. This is especially the case when sound prior information is available on part of the parameters, and the sample size is relatively small. Spatial dependence and false discovery rate are also discussed, and the parameter estimation is efficient. As an illustration, we used the proposed hybrid approach to analyze a real CGH data.     

Bayesian estimation of quantile distributions

Use of Bayesian modelling and analysis has become commonplace in many disciplines (finance, genetics and image analysis, for example). Many complex data sets are collected which do not readily admit standard distributions, and often comprise skew and kurtotic data. Such data is well-modelled by the very flexibly-shaped distributions of the quantile distribution family, whose members are defined by the inverse of their cumulative distribution functions and rarely have analytical likelihood functions defined. Without explicit likelihood functions, Bayesian methodologies such as Gibbs sampling cannot be applied to parameter estimation for this valuable class of distributions without resorting to numerical inversion. Approximate Bayesian computation provides an alternative approach requiring only a sampling scheme for the distribution of interest, enabling easier use of quantile distributions under the Bayesian framework. Parameter estimates for simulated and experimental data are presented.     

Geometric sample size determination in Bayesian analysis

The problem of sample size determination in the context of Bayesian analysis is considered. For the familiar and practically important parameter of a geometric distribution with a beta prior, three different Bayesian approaches based on the highest posterior density intervals are discussed. A computer program handles all computational complexities and is available upon request.     

Bayesian Recovery of the Initial Condition for the Heat Equation

We study a Bayesian approach to recovering the initial condition for the heat equation from noisy observations of the solution at a later time. We consider a class of prior distributions indexed by a parameter quantifying “smoothness” and show that the corresponding posterior distributions contract around the true parameter at a rate that depends on the smoothness of the true initial condition and the smoothness and scale of the prior. Correct combinations of these characteristics lead to the optimal minimax rate. One type of priors leads to a rate-adaptive Bayesian procedure. The frequentist coverage of credible sets is shown to depend on the combination of the prior and true parameter as well, with smoother priors leading to zero coverage and rougher priors to (extremely) conservative results. In the latter case, credible sets are much larger than frequentist confidence sets, in that the ratio of diameters diverges to infinity. The results are numerically illustrated by a simulated data example.     

Equality and inequality constrained multivariate linear models: Objective model selection using constrained posterior priors

In objective Bayesian model selection, a well-known problem is that standard non-informative prior distributions cannot be used to obtain a sensible outcome of the Bayes factor because these priors are improper. The use of a small part of the data, i.e., a training sample, to obtain a proper posterior prior distribution has become a popular method to resolve this issue and seems to result in reasonable outcomes of default Bayes factors, such as the intrinsic Bayes factor or a Bayes factor based on the empirical expected-posterior prior.     

The use of Jeffreys priors for the Student-t distribution

The Jeffreys-rule prior and the marginal independence Jeffreys prior are recently proposed in Fonseca et al. [Objective Bayesian analysis for the Student-t regression model, Biometrika 95 (2008), pp. 325–333] as objective priors for the Student-t regression model. The authors showed that the priors provide proper posterior distributions and perform favourably in parameter estimation. Motivated by a practical financial risk management application, we compare the performance of the two Jeffreys priors with other priors proposed in the literature in a problem of estimating high quantiles for the Student-t model with unknown degrees of freedom. Through an asymptotic analysis and a simulation study, we show that both Jeffreys priors perform better in using a specific quantile of the Bayesian predictive distribution to approximate the true quantile.