Intrinsic Priors for Testing Two Normal Means with Intrinsic Bayes Factors

ABSTRACT

In this article we consider the problem of comparing two normal means with unknown common variance using a Bayesian approach. Conventional Bayes factors with improper non informative priors are not well defined. The intrinsic Bayes factors are used to overcome such a difficulty. We derive intrinsic priors whose Bayes factors are asymptotically equivalent to the corresponding intrinsic Bayes factors. We illustrate our results with numerical examples.     

Bayesian inference for the randomly censored Weibull distribution

In this paper, we consider the Bayesian inference of the unknown parameters of the randomly censored Weibull distribution. A joint conjugate prior on the model parameters does not exist; we assume that the parameters have independent gamma priors. Since closed-form expressions for the Bayes estimators cannot be obtained, we use Lindley's approximation, importance sampling and Gibbs sampling techniques to obtain the approximate Bayes estimates and the corresponding credible intervals. A simulation study is performed to observe the behaviour of the proposed estimators. A real data analysis is presented for illustrative purposes.     

Comparison of testing procedures utilizing P-values and Bayes factors in some common situations

Bayesian alternatives to classical tests for several testing problems are considered. One-sided and two-sided sets of hypotheses are tested concerning an exponential parameter, a Binomial proportion, and a normal mean. Hierarchical Bayes and noninformative Bayes procedures are compared with the appropriate classical procedure, either the uniformly most powerful test or the likelihood ratio test, in the different situations. The hierarchical prior employed is the conjugate prior at the first stage with the mean being the test parameter and a noninformative prior at the second stage for the hyper parameter(s) of the first stage prior. Fair comparisons are attempted in which fair means the likelihood of making a type I error is approximately the same for the different testing procedures; once this condition is satisfied, the power of the different tests are compared, the larger the power, the better the test. This comparison is difficult in the two-sided case due to the unsurprising discrepancy between Bayesian and classical measures of evidence that have been discussed for years. The hierarchical Bayes tests appear to compete well with the typical classical test in the one-sided cases.     

Hierarchical bayes estimation of normal variances with application to a random effect model

The problem of simultaneously estimating p normal variances is investigated when the parameters are believed a priori to be similar in size. A hierarchical Bayes approach is employed and the resulting estimator is compared to common estimators used including one proposed by Box and Tiao (1973) using a Bayesian approach with a noninformative prior. The technique is then applied to estimate components of variance in the one way layout random effect model of the analysis of variance.     

Model determination for the variance component model using reference priors

For the balanced variance component model when the inference concerning intraclass correlation coefficient is of interest, Bayesian analysis is often appropriate. However, the question remains is to choose the appropriate prior. In this paper, we consider testing of the intraclass correlation coefficient under a default prior specification. Berger and Bernardo's (1992) On the development of the reference prior method. In: Bernardo, J.M., Berger, J.O., Dawid, A.P., Smith, A.F.M. (Eds.), Bayesian Statist. Vol. 4. Oxford University Press, London, pp. 35–60 reference priors are developed and are used to obtain the intrinsic Bayes factor (Berger and Pericchi, 1996) The intrinsic Bayes factor for model selection and prediction. J. Amer. statist. Assoc. 91, 109–122 for the nested models. Influence diagnostics using intrinsic Bayes factors are also developed. Finally, one simulated data is provided which illustrates the proposed methodology with appropriate simulation based on computational formulas. Then in order to overcome the difficulty in Bayesian computation, MCMC method, such as Gibbs sampler and Metropolis–Hastings algorithm, is employed.     

Reanalyzing Ultimatum Bargaining—Comparing Nondecreasing Curves Without Shape Constraints

We employ a hierarchical Bayesian method with exchangeable prior distributions to estimate and compare similar nondecreasing response curves. A Dirichlet process distribution is assigned to each of the response curves as a first stage prior. A second stage prior is then used to model the hyperparameters. We define parameters which will be used to compare the response curves. A Markov chain Monte Carlo method is applied to compute the resulting Bayesian estimates. To illustrate the methodology, we re-examine data from an experiment designed to test whether experimenter observation influences the ultimatum game. A major restriction of the original analysis was the shape constraint that the present technique allows us to greatly relax. We also consider independent priors and use Bayes factors to compare various models.     

Variance components estimation for the balanced random effects model under mixed prior distributions

For the balanced random effects models, when the variance components are correlated either naturally or through common prior structures, by assuming a mixed prior distribution for the variance components, we propose some new Bayesian estimators. To contrast and compare the new estimators with the minimum variance unbiased (MVUE) and restricted maximum likelihood estimators (RMLE), some simulation studies are also carried out. It turns out that the proposed estimators have smaller mean squared errors than the MVUE and RMLE.     

Equality and inequality constrained multivariate linear models: Objective model selection using constrained posterior priors

In objective Bayesian model selection, a well-known problem is that standard non-informative prior distributions cannot be used to obtain a sensible outcome of the Bayes factor because these priors are improper. The use of a small part of the data, i.e., a training sample, to obtain a proper posterior prior distribution has become a popular method to resolve this issue and seems to result in reasonable outcomes of default Bayes factors, such as the intrinsic Bayes factor or a Bayes factor based on the empirical expected-posterior prior.     

Approximate Bayes estimation of the parameters and reliability function of a mixture of two inverse Weibull distributions under Type-2 censoring

The main goal of this paper is to develop the approximate Bayes estimation of the five-dimensional vector of the parameters and reliability function of a mixture of two inverse Weibull distributions (MTIWD) under Type-2 censoring. Usually, the posterior distribution is complicated under the scheme of Type-2 censoring and the integrals that are involved cannot be obtained in a simple explicit form. In this study, we use Lindley's [Approximate Bayesian method, Trabajos Estadist. 31 (1980), pp. 223–237] approximate form of Bayes estimation in the case of an MTIWD under Type-2 censoring. Later, we calculate the estimated risks (ERs) of the Bayes estimates and compare them with the corresponding ERs of the maximum-likelihood estimates through Monte Carlo simulation. Finally, we analyse a real data set using the findings.     

Bayesian (mean) most powerful tests

A fundamental theorem in hypothesis testing is the Neyman‐Pearson (N‐P) lemma, which creates the most powerful test of simple hypotheses. In this article, we establish Bayesian framework of hypothesis testing, and extend the Neyman‐Pearson lemma to create the Bayesian most powerful test of general hypotheses, thus providing optimality theory to determine thresholds of Bayes factors. Unlike conventional Bayes tests, the proposed Bayesian test is able to control the type I error.     

Bayes Factor Consistency for One-way Random Effects Model

In this article, we consider Bayesian hypothesis testing for the balanced one-way random effects model. A special choice of the prior formulation for the ratio of variance components is shown to yield an explicit closed-form Bayes factor without integral representation. Furthermore, we study the consistency issue of the resulting Bayes factor under three asymptotic scenarios: either the number of units goes to infinity, the number of observations per unit goes to infinity, or both go to infinity. Finally, the behavior of the proposed approach is illustrated by simulation studies.     

Bayesian approach to change point problems

A Bayesian approach is considered to study the change point problems. A hypothesis for testing change versus no change is considered using the notion of predictive distributions. Bayes factors are developed for change versus no change in the exponential families of distributions with conjugate priors. Under vague prior information, both Bayes factors and pseudo Bayes factors are considered. A new result is developed which describes how the overall Bayes factor has a decomposition into Bayes factors at each point. Finally, an example is provided in which the computations are performed using the concept of imaginary observations.     

Random effects selection in generalized linear mixed models via shrinkage penalty function

In this paper, we discuss the selection of random effects within the framework of generalized linear mixed models (GLMMs). Based on a reparametrization of the covariance matrix of random effects in terms of modified Cholesky decomposition, we propose to add a shrinkage penalty term to the penalized quasi-likelihood (PQL) function of the variance components for selecting effective random effects. The shrinkage penalty term is taken as a function of the variance of random effects, initiated by the fact that if the variance is zero then the corresponding variable is no longer random (with probability one). The proposed method takes the advantage of a convenient computation for the PQL estimation and appealing properties for certain shrinkage penalty functions such as LASSO and SCAD. We propose to use a backfitting algorithm to estimate the fixed effects and variance components in GLMMs, which also selects effective random effects simultaneously. Simulation studies show that the proposed approach performs quite well in selecting effective random effects in GLMMs. Real data analysis is made using the proposed approach, too.     

Nonparametric hierarchical Bayes analysis of binomial data via Bernstein polynomial priors

For binomial data analysis, many methods based on empirical Bayes interpretations have been developed, in which a variance‐stabilizing transformation and a normality assumption are usually required. To achieve the greatest model flexibility, we conduct nonparametric Bayesian inference for binomial data and employ a special nonparametric Bayesian prior—the Bernstein–Dirichlet process (BDP)—in the hierarchical Bayes model for the data. The BDP is a special Dirichlet process (DP) mixture based on beta distributions, and the posterior distribution resulting from it has a smooth density defined on [0, 1]. We examine two Markov chain Monte Carlo procedures for simulating from the resulting posterior distribution, and compare their convergence rates and computational efficiency. In contrast to existing results for posterior consistency based on direct observations, the posterior consistency of the BDP, given indirect binomial data, is established. We study shrinkage effects and the robustness of the BDP‐based posterior estimators in comparison with several other empirical and hierarchical Bayes estimators, and we illustrate through examples that the BDP‐based nonparametric Bayesian estimate is more robust to the sample variation and tends to have a smaller estimation error than those based on the DP prior. In certain settings, the new estimator can also beat Stein's estimator, Efron and Morris's limited‐translation estimator, and many other existing empirical Bayes estimators. The Canadian Journal of Statistics 40: 328–344; 2012 © 2012 Statistical Society of Canada     

Bayesian inference for the randomly censored Burr-type XII distribution

The article presents the Bayesian inference for the parameters of randomly censored Burr-type XII distribution with proportional hazards. The joint conjugate prior of the proposed model parameters does not exist; we consider two different systems of priors for Bayesian estimation. The explicit forms of the Bayes estimators are not possible; we use Lindley's method to obtain the Bayes estimates. However, it is not possible to obtain the Bayesian credible intervals with Lindley's method; we suggest the Gibbs sampling procedure for this purpose. Numerical experiments are performed to check the properties of the different estimators. The proposed methodology is applied to a real-life data for illustrative purposes. The Bayes estimators are compared with the Maximum likelihood estimators via numerical experiments and real data analysis. The model is validated using posterior predictive simulation in order to ascertain its appropriateness.     

Bayesian model selection using test statistics

Summary.  Existing Bayesian model selection procedures require the specification of prior distributions on the parameters appearing in every model in the selection set. In practice, this requirement limits the application of Bayesian model selection methodology. To overcome this limitation, we propose a new approach towards Bayesian model selection that uses classical test statistics to compute Bayes factors between possible models. In several test cases, our approach produces results that are similar to previously proposed Bayesian model selection and model averaging techniques in which prior distributions were carefully chosen. In addition to eliminating the requirement to specify complicated prior distributions, this method offers important computational and algorithmic advantages over existing simulation-based methods. Because it is easy to evaluate the operating characteristics of this procedure for a given sample size and specified number of covariates, our method facilitates the selection of hyperparameter values through prior-predictive simulation.     

Objective Bayesian multiple comparisons for normal variances

This paper considers the multiple comparisons problem for normal variances. We propose a solution based on a Bayesian model selection procedure to this problem in which no subjective input is considered. We construct the intrinsic and fractional priors for which the Bayes factors and model selection probabilities are well defined. The posterior probability of each model is used as a model selection tool. The behaviour of these Bayes factors is compared with the Bayesian information criterion of Schwarz and some frequentist tests.     

BAYESIAN SUBSET SELECTION AND MODEL AVERAGING USING A CENTRED AND DISPERSED PRIOR FOR THE ERROR VARIANCE

This article proposes a new data‐based prior distribution for the error variance in a Gaussian linear regression model, when the model is used for Bayesian variable selection and model averaging. For a given subset of variables in the model, this prior has a mode that is an unbiased estimator of the error variance but is suitably dispersed to make it uninformative relative to the marginal likelihood. The advantage of this empirical Bayes prior for the error variance is that it is centred and dispersed sensibly and avoids the arbitrary specification of hyperparameters. The performance of the new prior is compared to that of a prior proposed previously in the literature using several simulated examples and two loss functions. For each example our paper also reports results for the model that orthogonalizes the predictor variables before performing subset selection. A real example is also investigated. The empirical results suggest that for both the simulated and real data, the performance of the estimators based on the prior proposed in our article compares favourably with that of a prior used previously in the literature.     

Improved robust Bayes estimators of the error variance in linear models

We consider the problem of estimating the error variance in a general linear model when the error distribution is assumed to be spherically symmetric, but not necessary Gaussian. In particular we study the case of a scale mixture of Gaussians including the particularly important case of the multivariate-t distribution. Under Stein's loss, we construct a class of estimators that improve on the usual best unbiased (and best equivariant) estimator. Our class has the interesting double robustness property of being simultaneously generalized Bayes (for the same generalized prior) and minimax over the entire class of scale mixture of Gaussian distributions.