Criticism of the predictive distribution

The predictive distribution is a mixture of the original distribution model and is used for predicting a future observation. Therein, the mixing distribution is the posterior distribution of the distribution parameters in the Bayesian inference. The mixture can also be computed for the frequentist inference because the Bayesian posterior distribution has the same meaning as a frequentist confidence interval. I present arguments against the concept of predictive distribution. Examples illustrate these. The most important argument is that the predictive distribution can depend on the parameterization. An improvement of the theory of the predictive distribution is recommended.     

On predictive distributions and Bayesian networks

In this paper we are interested in discrete prediction problems for a decision-theoretic setting, where the task is to compute the predictive distribution for a finite set of possible alternatives. This question is first addressed in a general Bayesian framework, where we consider a set of probability distributions defined by some parametric model class. Given a prior distribution on the model parameters and a set of sample data, one possible approach for determining a predictive distribution is to fix the parameters to the instantiation with the maximum a posteriori probability. A more accurate predictive distribution can be obtained by computing the evidence (marginal likelihood), i.e., the integral over all the individual parameter instantiations. As an alternative to these two approaches, we demonstrate how to use Rissanen's new definition of stochastic complexity for determining predictive distributions, and show how the evidence predictive distribution with Jeffrey's prior approaches the new stochastic complexity predictive distribution in the limit with increasing amount of sample data. To compare the alternative approaches in practice, each of the predictive distributions discussed is instantiated in the Bayesian network model family case. In particular, to determine Jeffrey's prior for this model family, we show how to compute the (expected) Fisher information matrix for a fixed but arbitrary Bayesian network structure. In the empirical part of the paper the predictive distributions are compared by using the simple tree-structured Naive Bayes model, which is used in the experiments for computational reasons. The experimentation with several public domain classification datasets suggest that the evidence approach produces the most accurate predictions in the log-score sense. The evidence-based methods are also quite robust in the sense that they predict surprisingly well even when only a small fraction of the full training set is used.     

Observations on the effect of the prior distribution on the predictive distribution in Bayesian inferences

Typically, in the brief discussion of Bayesian inferential methods presented at the beginning of calculus-based undergraduate or graduate mathematical statistics courses, little attention is paid to the process of choosing the parameter value(s) for the prior distribution. Even less attention is paid to the impact of these choices on the predictive distribution of the data. Reasons for this include that the posterior can be found by ignoring the predictive distribution thereby streamlining the derivation of the posterior and/or that computer software can be used to find the posterior distribution. In this paper, the binomial, negative-binomial and Poisson distributions along with their conjugate beta and gamma priors are utilized to obtain the resulting predictive distributions. It is then demonstrated that specific choices of the parameters of the priors can lead to predictive distributions with properties that might be surprising to a non-expert user of Bayesian methods.     

Dropping observations without affecting posterior and predictive distributions

Suppose in a distribution problem, the sample information W is split into two pieces W ₁ and W ₂, and the parameters involved are split into two sets, π containing the parameters of interest, and θ containing nuisance parameters. It is shown that, under certain conditions, the posterior distribution of π does not depend on the data W ₂, which can thus be ignored. This also has consequences for the predictive distribution of future (or missing) observations. In fact, under similar conditions, the predictive distributions using W or just W ₁ are identical.     

Bayesian Point Estimation Using the Predictive Distribution

There is considerable question about how a Bayesian might provide a point estimate for a parameter when no loss function is specified. The mean, median, and mode of the posterior distribution have all been suggested. This article considers a natural Bayesian estimator based on the predictive distribution of future observations. It is shown that for the set of parameters that admit an unbiased estimate, this predictive estimate coincides with the posterior mean of the parameter. It is argued that this result provides some justification for use of the posterior mean as a Bayesian point estimate when there is no loss structure.     

Bayesian inferences and forecasts with moving averages processes

This paper will develop Bayesian inferential and forecasting techniques which can be used with any moving average process. By employing the conditional likelihood function, at-approximation to the predictive distribution and the marginal posterior distribution of the moving average parameters is developed. Several examples demonstrate posterior and predictive inferences.     

Bayes prediction of number of failures in weibull samples

This work is concerned with the Bayesian prediction problem of the number of components which will fail in a future time interval, when the failure times are Weibull distributed. Both the 1-sample and the 2-sample prediction problems are dealed with, and some choices of the prior densities on the distribution parameters are discussed which are relatively easy to work with and allow different degrees of knowledge on the failure mechanism to be incorporated in the predictive procedure. Useful relations between the predictive distribution on the number of future failures and the predictive distribution on the future failure times are derived. Numerical examples are also given.     

Bayesian predictive power: choice of prior and some recommendations for its use as probability of success in drug development

Bayesian predictive power, the expectation of the power function with respect to a prior distribution for the true underlying effect size, is routinely used in drug development to quantify the probability of success of a clinical trial. Choosing the prior is crucial for the properties and interpretability of Bayesian predictive power. We review recommendations on the choice of prior for Bayesian predictive power and explore its features as a function of the prior. The density of power values induced by a given prior is derived analytically and its shape characterized. We find that for a typical clinical trial scenario, this density has a u‐shape very similar, but not equal, to a β‐distribution. Alternative priors are discussed, and practical recommendations to assess the sensitivity of Bayesian predictive power to its input parameters are provided. Copyright © 2016 John Wiley & Sons, Ltd.     

A matching prior for extreme quantile estimation of the generalized Pareto distribution

Extreme quantile estimation plays an important role in risk management and environmental statistics among other applications. A popular method is the peaks-over-threshold (POT) model that approximate the distribution of excesses over a high threshold through generalized Pareto distribution (GPD). Motivated by a practical financial risk management problem, we look for an appropriate prior choice for Bayesian estimation of the GPD parameters that results in better quantile estimation. Specifically, we propose a noninformative matching prior for the parameters of a GPD so that a specific quantile of the Bayesian predictive distribution matches the true quantile in the sense of Datta et al. (2000).     

Inference and prediction with arma processes

An essential ingredient of any time series analysis is the estimation of the model parameters and the forecasting of future observations. This investigation takes a Bayesian approach to the analysis of time series by making inferences of the model parameters from the posterior distribution and forecasting from the predictive distribution.

The foundation of the approach is to approximate the condi-tional likelihood by a normal-gamma distribution on the parameter space. The techniques illustrated with many examples of ARMA processes.     

Sampling the Future: A Bayesian Approach to Forecasting From Univariate Time Series Models

The Box–Jenkins methodology for modeling and forecasting from univariate time series models has long been considered a standard to which other forecasting techniques have been compared. To a Bayesian statistician, however, the method lacks an important facet—a provision for modeling uncertainty about parameter estimates. We present a technique called sampling the future for including this feature in both the estimation and forecasting stages. Although it is relatively easy to use Bayesian methods to estimate the parameters in an autoregressive integrated moving average (ARIMA) model, there are severe difficulties in producing forecasts from such a model. The multiperiod predictive density does not have a convenient closed form, so approximations are needed. In this article, exact Bayesian forecasting is approximated by simulating the joint predictive distribution. First, parameter sets are randomly generated from the joint posterior distribution. These are then used to simulate future paths of the time series. This bundle of many possible realizations is used to project the future in several ways. Highest probability forecast regions are formed and portrayed with computer graphics. The predictive density's shape is explored. Finally, we discuss a method that allows the analyst to subjectively modify the posterior distribution on the parameters and produce alternate forecasts.     

Bayesian parameter estimation via variational methods

We consider a logistic regression model with a Gaussian prior distribution over the parameters. We show that an accurate variational transformation can be used to obtain a closed form approximation to the posterior distribution of the parameters thereby yielding an approximate posterior predictive model. This approach is readily extended to binary graphical model with complete observations. For graphical models with incomplete observations we utilize an additional variational transformation and again obtain a closed form approximation to the posterior. Finally, we show that the dual of the regression problem gives a latent variable density model, the variational formulation of which leads to exactly solvable EM updates.     

Bayesian analysis of threshold autoregressions

A nonasymptotic Bayesian approach is developed for analysis of data from threshold autoregressive processes with two regimes. Using the conditional likelihood function, the marginal posterior distribution for each of the parameters is derived along with posterior means and variances. A test for linear functions of the autoregressive coefficients is presented. The approach presented uses a posterior p-value averaged over the values of the threshold. The one-step ahead predictive distribution is derived along with the predictive mean and variance. In addition, equivalent results are derived conditional upon a value of the threshold. A numerical example is presented to illustrate the approach.     

ESTIMATION, PREDICTION AND INFERENCE FOR THE LASSO RANDOM EFFECTS MODEL

The least absolute shrinkage and selection operator (LASSO) can be formulated as a random effects model with an associated variance parameter that can be estimated with other components of variance. In this paper, estimation of the variance parameters is performed by means of an approximation to the marginal likelihood of the observed outcomes. The approximation is based on an alternative but equivalent formulation of the LASSO random effects model. Predictions can be made using point summaries of the predictive distribution of the random effects given the data with the parameters set to their estimated values. The standard LASSO method uses the mode of this distribution as the predictor. It is not the only choice, and a number of other possibilities are defined and empirically assessed in this article. The predictive mode is competitive with the predictive mean (best predictor), but no single predictor performs best across in all situations. Inference for the LASSO random effects is performed using predictive probability statements, which are more appropriate under the random effects formulation than tests of hypothesis.     

Estimation of random coefficient regression models

Linear regression models with coefficients across individual units regarded as random samples from some population are studied in this article from a Bayesian viewpoint. A prior distribution of the secondary parameters is derived following the Jeffreys rule. Posterior distribution of the primary and secondary parameters, and the predictive distribution of the future value are then examined. Computations of the parameter estimates are found to be rather straightforward. Data from a performance test on pigs is analysed and discussed. We also discuss the difficulties involved in using a Lindley and Smith (1972) prior in this problem.     

Option pricing under model and parameter uncertainty using predictive densities

The theoretical price of a financial option is given by the expectation of its discounted expiry time payoff. The computation of this expectation depends on the density of the value of the underlying instrument at expiry time. This density depends on both the parametric model assumed for the behaviour of the underlying, and the values of parameters within the model, such as volatility. However neither the model, nor the parameter values are known. Common practice when pricing options is to assume a specific model, such as geometric Brownian Motion, and to use point estimates of the model parameters, thereby precisely defining a density function.We explicitly acknowledge the uncertainty of model and parameters by constructing the predictive density of the underlying as an average of model predictive densities, weighted by each model's posterior probability. A model's predictive density is constructed by integrating its transition density function by the posterior distribution of its parameters. This is an extension to Bayesian model averaging. Sampling importance-resampling and Monte Carlo algorithms implement the computation. The advantage of this method is that rather than falsely assuming the model and parameter values are known, inherent ignorance is acknowledged and dealt with in a mathematically logical manner, which utilises all information from past and current observations to generate and update option prices. Moreover point estimates for parameters are unnecessary. We use this method to price a European Call option on a share index.     

Robust linear mixed models for Small Area Estimation

Hierarchical models are popular in many applied statistics fields including Small Area Estimation. One well known model employed in this particular field is the Fay–Herriot model, in which unobservable parameters are assumed to be Gaussian. In Hierarchical models assumptions about unobservable quantities are difficult to check. For a special case of the Fay–Herriot model, Sinharay and Stern [2003. Posterior predictive model checking in Hierarchical models. J. Statist. Plann. Inference 111, 209–221] showed that violations of the assumptions about the random effects are difficult to detect using posterior predictive checks. In this present paper we consider two extensions of the Fay–Herriot model in which the random effects are assumed to be distributed according to either an exponential power (EP) distribution or a skewed EP distribution. We aim to explore the robustness of the Fay–Herriot model for the estimation of individual area means as well as the empirical distribution function of their ‘ensemble’. Our findings, which are based on a simulation experiment, are largely consistent with those of Sinharay and Stern as far as the efficient estimation of individual small area parameters is concerned. However, when estimating the empirical distribution function of the ‘ensemble’ of small area parameters, results are more sensitive to the failure of distributional assumptions.     

A comparison of nonparametric priors in hierarchical mixture modelling for AFT regression

We will pursue a Bayesian nonparametric approach in the hierarchical mixture modelling of lifetime data in two situations: density estimation, when the distribution is a mixture of parametric densities with a nonparametric mixing measure, and accelerated failure time (AFT) regression modelling, when the same type of mixture is used for the distribution of the error term. The Dirichlet process is a popular choice for the mixing measure, yielding a Dirichlet process mixture model for the error; as an alternative, we also allow the mixing measure to be equal to a normalized inverse-Gaussian prior, built from normalized inverse-Gaussian finite dimensional distributions, as recently proposed in the literature. Markov chain Monte Carlo techniques will be used to estimate the predictive distribution of the survival time, along with the posterior distribution of the regression parameters. A comparison between the two models will be carried out on the grounds of their predictive power and their ability to identify the number of components in a given mixture density.     

A BAYESIAN METHOD FOR THE CHOICE OF THE SAMPLE SIZE IN EQUIVALENCE TRIALS

In this paper we consider a Bayesian predictive approach to sample size determination in equivalence trials. Equivalence experiments are conducted to show that the unknown difference between two parameters is small. For instance, in clinical practice this kind of experiment aims to determine whether the effects of two medical interventions are therapeutically similar. We declare an experiment successful if an interval estimate of the effects‐difference is included in a set of values of the parameter of interest indicating a negligible difference between treatment effects (equivalence interval). We derive two alternative criteria for the selection of the optimal sample size, one based on the predictive expectation of the interval limits and the other based on the predictive probability that these limits fall in the equivalence interval. Moreover, for both criteria we derive a robust version with respect to the choice of the prior distribution. Numerical results are provided and an application is illustrated when the normal model with conjugate prior distributions is assumed.     

On the construction of bayesian prediction limits for the weibull distribution

This paper is concerned with a BAYESian construction of the prediction limits for the Weibull distribution as an example of extreme value distributions. Thus, considering Weibull and Uniform distributions for the parameters, the predictive functions, which may lead to approximative evaluation of the prediction limits, is determined by using simulation methods