A note on nonexistence of posterior moments

Bayesian analyses often take for granted the assumption that the posterior distribution has at least a first moment. They often include computed or estimated posterior means. In this note, the authors show an example of a Weibull distribution parameter where the theoretical posterior mean fails to exist for commonly used proper semi–conjugate priors. They also show that posterior moments can fail to exist with commonly used noninformative priors including Jeffreys, reference and matching priors, despite the fact that the posteriors are proper. Moreover, within a broad class of priors, the predictive distribution also has no mean. The authors illustrate the problem with a simulated example. Their results demonstrate that the unwitting use of estimated posterior means may yield unjustified conclusions.     

Estimation of covariance matrices based on hierarchical inverse-Wishart priors

This paper focuses on Bayesian shrinkage methods for covariance matrix estimation. We examine posterior properties and frequentist risks of Bayesian estimators based on new hierarchical inverse-Wishart priors. More precisely, we give the conditions for the existence of the posterior distributions. Advantages in terms of numerical simulations of posteriors are shown. A simulation study illustrates the performance of the estimation procedures under three loss functions for relevant sample sizes and various covariance structures.     

Bayesian shrinkage estimates for regression coefficients in m populations

For a linear regression model over m populations with separate regression coefficients but a common error variance, a Bayesian model is employed to obtain regression coefficient estimates which are shrunk toward an overall value. The formulation uses Normal priors on the coefficients and diffuse priors on the grand mean vectors, the error variance, and the between-to-error variance ratios. The posterior density of the parameters which were given diffuse priors is obtained. From this the posterior means and variances of regression coefficients and the predictive mean and variance of a future observation are obtained directly by numerical integration in the balanced case, and with the aid of series expansions in the approximately balanced case. An example is presented and worked out for the case of one predictor variable. The method is an extension of Box & Tiao's Bayesian estimation of means in the balanced one-way random effects model.     

Objective Bayesian analysis of spatial data with measurement error

The author shows how geostatistical data that contain measurement errors can be analyzed objectively by a Bayesian approach using Gaussian random fields. He proposes a reference prior and two versions of Jeffreys' prior for the model parameters. He studies the propriety and the existence of moments for the resulting posteriors. He also establishes the existence of the mean and variance of the predictive distributions based on these default priors. His reference prior derives from a representation of the integrated likelihood that is particularly convenient for computation and analysis. He further shows that these default priors are not very sensitive to some aspects of the design and model, and that they have good frequentist properties. Finally, he uses a data set of carbon/nitrogen ratios from an agricultural field to illustrate his approach.     

Bayesian approach to estimation of intraclass correlation using reference prior

For the balanced variance component model when the intraclass correlation coefficient is of interest, Bayesian analysis is often appropriate. Berger and Bernardo’s (1992a) grouped ordering reference prior approach is used to analyze this model. The reference priors are developed and compared for the posterior inference with real and simulated data. We examine whether the reference priors satisfy the probability-matching criterion. Further, the reference prior is shown to be good in the sense of correct frequentist coverage probability of the posterior quantile.     

Bayesian analysis of the generalized gamma distribution using non-informative priors

The Generalized gamma (GG) distribution plays an important role in statistical analysis. For this distribution, we derive non-informative priors using formal rules, such as Jeffreys prior, maximal data information prior and reference priors. We have shown that these most popular formal rules with natural ordering of parameters, lead to priors with improper posteriors. This problem is overcome by considering a prior averaging approach discussed in Berger et al. [Overall objective priors. Bayesian Analysis. 2015;10(1):189–221]. The obtained hybrid Jeffreys-reference prior is invariant under one-to-one transformations and yields a proper posterior distribution. We obtained good frequentist properties of the proposed prior using a detailed simulation study. Finally, an analysis of the maximum annual discharge of the river Rhine at Lobith is presented.     

On conjugate families and Jeffreys priors for von Mises–Fisher distributions

This paper discusses characteristics of standard conjugate priors and their induced posteriors in Bayesian inference for von Mises–Fisher distributions, using either the canonical natural exponential family or the more commonly employed polar coordinate parameterizations. We analyze when standard conjugate priors as well as posteriors are proper, and investigate the Jeffreys prior for the von Mises–Fisher family. Finally, we characterize the proper distributions in the standard conjugate family of the (matrix-valued) von Mises–Fisher distributions on Stiefel manifolds.     

Optimal collapsing of mixture distributions in robust recursive estimation

Several authors have discussed Kalman filtering procedures using a mixture of normals as a model for the distributions of the noise in the observation and/or the state space equations. Under this model, resulting posteriors involve a mixture of normal distributions, and a “collapsing method” must be found in order to keep the recursive procedure simple. We prove that the Kullback-Leibler distance between the mixture posterior and that of a single normal distribution is minimized when we choose the mean and variance of the single normal distribution to be the mean and variance of the mixture posterior. Hence, “collapsing by moments” is optimal in this sense. We then develop the resulting optimal algorithm for “Kalman filtering” for this situation, and illustrate its performance with an example.     

Bayesian Estimation of the Parameters of Bivariate Exponential Distributions

In this article, we develop an empirical Bayesian approach for the Bayesian estimation of parameters in four bivariate exponential (BVE) distributions. We have opted for gamma distribution as a prior for the parameters of the model in which the hyper parameters have been estimated based on the method of moments and maximum likelihood estimates (MLEs). A simulation study was conducted to compute empirical Bayesian estimates of the parameters and their standard errors. We use moment estimators or MLEs to estimate the hyper parameters of the prior distributions. Furthermore, we compare the posterior mode of parameters obtained by different prior distributions and the Bayesian estimates based on gamma priors are very close to the true values as compared to improper priors. We use MCMC method to obtain the posterior mean and compared the same using the improper priors and the classical estimates, MLEs.     

Bayesian regularisation in structured additive regression: a unifying perspective on shrinkage, smoothing and predictor selection

This paper surveys various shrinkage, smoothing and selection priors from a unifying perspective and shows how to combine them for Bayesian regularisation in the general class of structured additive regression models. As a common feature, all regularisation priors are conditionally Gaussian, given further parameters regularising model complexity. Hyperpriors for these parameters encourage shrinkage, smoothness or selection. It is shown that these regularisation (log-) priors can be interpreted as Bayesian analogues of several well-known frequentist penalty terms. Inference can be carried out with unified and computationally efficient MCMC schemes, estimating regularised regression coefficients and basis function coefficients simultaneously with complexity parameters and measuring uncertainty via corresponding marginal posteriors. For variable and function selection we discuss several variants of spike and slab priors which can also be cast into the framework of conditionally Gaussian priors. The performance of the Bayesian regularisation approaches is demonstrated in a hazard regression model and a high-dimensional geoadditive regression model.     

Bayesian estimation for first-order autoregressive model with explanatory variables

In this article, we develop a Bayesian analysis in autoregressive model with explanatory variables. When σ² is known, we consider a normal prior and give the Bayesian estimator for the regression coefficients of the model. For the case σ² is unknown, another Bayesian estimator is given for all unknown parameters under a conjugate prior. Bayesian model selection problem is also being considered under the double-exponential priors. By the convergence of ρ-mixing sequence, the consistency and asymptotic normality of the Bayesian estimators of the regression coefficients are proved. Simulation results indicate that our Bayesian estimators are not strongly dependent on the priors, and are robust.     

On Bayesian estimators in multistage binomial designs

A new class of Bayesian estimators for a proportion in multistage binomial designs is considered. Priors belong to the beta-J distribution family, which is derived from the Fisher information associated with the design. The transposition of the beta parameters of the Haldane and the uniform priors in fixed binomial experiments into the beta-J distribution yields bias-corrected versions of these priors in multistage designs. We show that the estimator of the posterior mean based on the corrected Haldane prior and the estimator of the posterior mode based on the corrected uniform prior have good frequentist properties. An easy-to-use approximation of the estimator of the posterior mode is provided. The new Bayesian estimators are compared to Whitehead's and the uniformly minimum variance estimators through several multistage designs. Last, the bias of the estimator of the posterior mode is derived for a particular case.     

Bayesian analysis of vector-autoregressive models with noninformative priors

In this paper, we investigate the properties of Bayes estimators of vector autoregression (VAR) coefficients and the covariance matrix under two commonly employed loss functions. We point out that the posterior mean of the variances of the VAR errors under the Jeffreys prior is likely to have an over-estimation bias. Our Bayesian computation results indicate that estimates using the constant prior on the VAR regression coefficients and the reference prior of Yang and Berger (Ann. Statist. 22 (1994) 1195) on the covariance matrix dominate the constant-Jeffreys prior estimates commonly used in applications of VAR models in macroeconomics. We also estimate a VAR model of consumption growth using both constant-reference and constant-Jeffreys priors.     

A prior for the variance in hierarchical models

The choice of prior distributions for the variances can be important and quite difficult in Bayesian hierarchical and variance component models. For situations where little prior information is available, a ‘nonin-formative’ type prior is usually chosen. ‘Noninformative’ priors have been discussed by many authors and used in many contexts. However, care must be taken using these prior distributions as many are improper and thus, can lead to improper posterior distributions. Additionally, in small samples, these priors can be ‘informative’. In this paper, we investigate a proper ‘vague’ prior, the uniform shrinkage prior (Strawder-man 1971; Christiansen & Morris 1997). We discuss its properties and show how posterior distributions for common hierarchical models using this prior lead to proper posterior distributions. We also illustrate the attractive frequentist properties of this prior for a normal hierarchical model including testing and estimation. To conclude, we generalize this prior to the multivariate situation of a covariance matrix.     

Inference with three-level prior distributions in quantile regression problems

In this paper, we propose a three level hierarchical Bayesian model for variable selection and estimation in quantile regression problems. Specifically, at the first level we consider a zero mean normal priors for the coefficients with unknown variance parameters. At the second level, we specify two different priors for the unknown variance parameters which introduce two different models producing different levels of sparsity. Then, at the third level we suggest joint improper priors for the unknown hyperparameters assuming they are independent. Simulations and Boston Housing data are utilized to compare the performance of our models with six existing models. The results indicate that our models perform good in the simulations and Boston Housing data.     

Bayesian predictive densities based on latent information priors

Construction methods for prior densities are investigated from a predictive viewpoint. Predictive densities for future observables are constructed by using observed data. The simultaneous distribution of future observables and observed data is assumed to belong to a parametric submodel of a multinomial model. Future observables and data are possibly dependent. The discrepancy of a predictive density to the true conditional density of future observables given observed data is evaluated by the Kullback-Leibler divergence. It is proved that limits of Bayesian predictive densities form an essentially complete class. Latent information priors are defined as priors maximizing the conditional mutual information between the parameter and the future observables given the observed data. Minimax predictive densities are constructed as limits of Bayesian predictive densities based on prior sequences converging to the latent information priors.     

Noninformative priors for linear combinations of normal means with unequal variances

For normal populations with unequal variances, we develop matching priors and reference priors for a linear combination of the means. Here, we find three second-order matching priors: a highest posterior density (HPD) matching prior, a cumulative distribution function (CDF) matching prior, and a likelihood ratio (LR) matching prior. Furthermore, we show that the reference priors are all first-order matching priors, but that they do not satisfy the second-order matching criterion that establishes the symmetry and the unimodality of the posterior under the developed priors. The results of a simulation indicate that the second-order matching prior outperforms the reference priors in terms of matching the target coverage probabilities, in a frequentist sense. Finally, we compare the Bayesian credible intervals based on the developed priors with the confidence intervals derived from real data.     

An efficient Monte Carlo EM algorithm for Bayesian lasso

The lasso is a popular technique of simultaneous estimation and variable selection in many research areas. The marginal posterior mode of the regression coefficients is equivalent to estimates given by the non-Bayesian lasso when the regression coefficients have independent Laplace priors. Because of its flexibility of statistical inferences, the Bayesian approach is attracting a growing body of research in recent years. Current approaches are primarily to either do a fully Bayesian analysis using Markov chain Monte Carlo (MCMC) algorithm or use Monte Carlo expectation maximization (MCEM) methods with an MCMC algorithm in each E-step. However, MCMC-based Bayesian method has much computational burden and slow convergence. Tan et al. [An efficient MCEM algorithm for fitting generalized linear mixed models for correlated binary data. J Stat Comput Simul. 2007;77:929–943] proposed a non-iterative sampling approach, the inverse Bayes formula (IBF) sampler, for computing posteriors of a hierarchical model in the structure of MCEM. Motivated by their paper, we develop this IBF sampler in the structure of MCEM to give the marginal posterior mode of the regression coefficients for the Bayesian lasso, by adjusting the weights of importance sampling, when the full conditional distribution is not explicit. Simulation experiments show that the computational time is much reduced with our method based on the expectation maximization algorithm and our algorithms and our methods behave comparably with other Bayesian lasso methods not only in prediction accuracy but also in variable selection accuracy and even better especially when the sample size is relatively large.     

Approximate reference priors for Gaussian random fields

Reference priors are theoretically attractive for the analysis of geostatistical data since they enable automatic Bayesian analysis and have desirable Bayesian and frequentist properties. But their use is hindered by computational hurdles that make their application in practice challenging. In this work, we derive a new class of default priors that approximate reference priors for the parameters of some Gaussian random fields. It is based on an approximation to the integrated likelihood of the covariance parameters derived from the spectral approximation of stationary random fields. This prior depends on the structure of the mean function and the spectral density of the model evaluated at a set of spectral points associated with an auxiliary regular grid. In addition to preserving the desirable Bayesian and frequentist properties, these approximate reference priors are more stable, and their computations are much less onerous than those of exact reference priors. Unlike exact reference priors, the marginal approximate reference prior of correlation parameter is always proper, regardless of the mean function or the smoothness of the correlation function. This property has important consequences for covariance model selection. An illustration comparing default Bayesian analyses is provided with a dataset of lead pollution in Galicia, Spain.     

The use of local and nonlocal priors in Bayesian test-based monitoring for single-arm phase II clinical trials

Bayesian sequential monitoring is widely used in adaptive phase II studies where the objective is to examine whether an experimental drug is efficacious. Common approaches for Bayesian sequential monitoring are based on posterior or predictive probabilities and Bayesian hypothesis testing procedures using Bayes factors. In the first part of the paper, we briefly show the connections between test-based (TB) and posterior probability-based (PB) sequential monitoring approaches. Next, we extensively investigate the choice of local and nonlocal priors for the TB monitoring procedure. We describe the pros and cons of different priors in terms of operating characteristics. We also develop a user-friendly Shiny application to implement the TB design.