Alternative Bayes factors for model selection

Several alternative Bayes factors have been recently proposed in order to solve the problem of the extreme sensitivity of the Bayes factor to the priors of models under comparison. Specifically, the impossibility of using the Bayes factor with standard noninformative priors for model comparison has led to the introduction of new automatic criteria, such as the posterior Bayes factor (Aitkin 1991), the intrinsic Bayes factors (Berger and Pericchi 1996b) and the fractional Bayes factor (O'Hagan 1995). We derive some interesting properties of the fractional Bayes factor that provide justifications for its use additional to the ones given by O'Hagan. We further argue that the use of the fractional Bayes factor, originally introduced to cope with improper priors, is also useful in a robust analysis. Finally, using usual classes of priors, we compare several alternative Bayes factors for the problem of testing the point null hypothesis in the univariate normal model.     

Alternative Procedures to Discriminate Non Nested Multivariate Linear Regression Models

ABSTRACT

This article builds classical and Bayesian testing procedures for choosing between non nested multivariate regression models. Although there are several classical tests for discriminating univariate regressions, only the Cox test is able to consistently handle the multivariate case. We then derive the limiting distribution of the Cox statistic in such a context, correcting an earlier derivation in the literature. Further, we show how to build alternative Bayes factors for the testing of nonnested multivariate linear regression models. In particular, we compute expressions for the posterior Bayes factor, the fractional Bayes factor, and the intrinsic Bayes factor.     

On asymptotic properties of Bayesian partially linear models

In this paper, we present large sample properties of a partially linear model from the Bayesian perspective, in which responses are explained by the semiparametric regression model with the additive form of the linear component and the nonparametric component. For this purpose, we investigate asymptotic behaviors of posterior distributions in terms of consistency. Specifically, we deal with a specific Bayesian partially linear regression model with additive noises in which the nonparametric component is modeled using Gaussian process priors. Under the Bayesian partially linear model using Gaussian process priors, we focus on consistency of posterior distribution and consistency of the Bayes factor, and extend these results to generalized additive regression models and study their asymptotic properties. In addition we illustrate the asymptotic properties based on empirical analysis through simulation studies.     

Time series models associated with Mittag-Leffler type distributions and its properties

ABSTRACT

We have provided a fractional generalization of the Poisson renewal processes by replacing the first time derivative in the relaxation equation of the survival probability by a fractional derivative of order α(0 < α ? 1). A generalized Laplacian model associated with the Mittag-Leffler distribution is examined. We also discuss some properties of this new model and its relevance to time series. Distribution of gliding sums, regression behaviors, and sample path properties are studied. Finally we introduce the q-Mittag-Leffler process associated with the q-Mittag-Leffler distribution.     

INTRINSIC PRIORS FOR TWO-SAMPLE TESTS IN NORMAL POPULATIONS

ABSTRACT

There have been considerable amounts of work regarding the development of various default Bayes factors in model selection and hypothesis testing. Two commonly used criteria, the intrinsic Bayes factor and the fractional Bayes factor are compared to test two independent normal means and variances. We also derive several intrinsic priors whose Bayes factors are asymptotically equivalent to the respective Bayes factors. We demonstrate our results in simulated datasets.     

Consistent fractional Bayes factor for nested normal linear models

In the Bayesian approach to parametric model comparison, the use of improper priors is problematic due to the indeterminacy of the resulting Bayes factor (BF). The need for developing automatic and robust methods for model comparison has led to the introduction of alternative BFs. Intrinsic Bayes factors (Berger and Pericchi, 1996a) and fractional Bayes factors (FBF) (O'Hagan, 1995) are two alternative strategies for default model selection. We show in this paper that the FBF can be inconsistent. To overcome this problem, we propose a generalization of the FBF approach that leads to the usual FBF or to some variants of it in some special cases. As an important problem, we consider and discuss this generalization for model selection in nested linear models.     

A Generalized Bayes Rule for Prediction

In the case of prior knowledge about the unknown parameter, the Bayesian predictive density coincides with the Bayes estimator for the true density in the sense of the Kullback-Leibler divergence, but this is no longer true if we consider another loss function. In this paper we present a generalized Bayes rule to obtain Bayes density estimators with respect to any α-divergence, including the Kullback-Leibler divergence and the Hellinger distance. For curved exponential models, we study the asymptotic behaviour of these predictive densities. We show that, whatever prior we use, the generalized Bayes rule improves (in a non-Bayesian sense) the estimative density corresponding to a bias modification of the maximum likelihood estimator. It gives rise to a correspondence between choosing a prior density for the generalized Bayes rule and fixing a bias for the maximum likelihood estimator in the classical setting. A criterion for comparing and selecting prior densities is also given.     

A ROBUST BAYES FACTOR FOR LINEAR MODELS

This paper proposes a new robust Bayes factor for comparing two linear models. The factor is based on a pseudo‐model for outliers and is more robust to outliers than the Bayes factor based on the variance‐inflation model for outliers. If an observation is considered an outlier for both models this new robust Bayes factor equals the Bayes factor calculated after removing the outlier. If an observation is considered an outlier for one model but not the other then this new robust Bayes factor equals the Bayes factor calculated without the observation, but a penalty is applied to the model considering the observation as an outlier. For moderate outliers where the variance‐inflation model is suitable, the two Bayes factors are similar. The new Bayes factor uses a single robustness parameter to describe a priori belief in the likelihood of outliers. Real and synthetic data illustrate the properties of the new robust Bayes factor and highlight the inferior properties of Bayes factors based on the variance‐inflation model for outliers.     

A pooled Bayes test of independence for sparse contingency tables from small areas

We study the association between bone mineral density (BMD) and body mass index (BMI) when contingency tables are constructed from the several U.S. counties, where BMD has three levels (normal, osteopenia and osteoporosis) and BMI has four levels (underweight, normal, overweight and obese). We use the Bayes factor (posterior odds divided by prior odds or equivalently the ratio of the marginal likelihoods) to construct the new test. Like the chi-squared test and Fisher's exact test, we have a direct Bayes test which is a standard test using data from each county. In our main contribution, for each county techniques of small area estimation are used to borrow strength across counties and a pooled test of independence of BMD and BMI is obtained using a hierarchical Bayesian model. Our pooled Bayes test is computed by performing a Monte Carlo integration using random samples rather than Gibbs samples. We have seen important differences among the pooled Bayes test, direct Bayes test and the Cressie-Read test that allows for some degree of sparseness, when the degree of evidence against independence is studied. As expected, we also found that the direct Bayes test is sensitive to the prior specifications but the pooled Bayes test is not so sensitive. Moreover, the pooled Bayes test has competitive power properties, and it is superior when the cell counts are small to moderate.     

Bayes admissible estimation of the means in Poisson decomposable graphical models

We investigate the Bayes estimation of the means in Poisson decomposable graphical models. Some classes of Bayes estimators are provided which improve on the maximum likelihood estimator under the normalized squared error loss. Both proper and improper priors are included in the proposed classes of priors. Concerning the generalized Bayes estimators with respect to the improper priors, we address their admissibility.     

A Bayesian Non-parametric Comparison of Two Treatments

In this paper we present a Bayesian non-parametric analysis of survival time data, involving information from two types of treatment. We present an easy to compute Bayes factor comparing two model assumptions: no treatment difference and treatment difference and use this to model summaries for each of the two treatments, in particular predictive distributions.     

Default Bayesian analysis of the Behrens–Fisher problem

In the Bayesian approach, the Behrens–Fisher problem has been posed as one of estimation for the difference of two means. No Bayesian solution to the Behrens–Fisher testing problem has yet been given due, perhaps, to the fact that the conventional priors used are improper. While default Bayesian analysis can be carried out for estimation purposes, it poses difficulties for testing problems. This paper generates sensible intrinsic and fractional prior distributions for the Behrens–Fisher testing problem from the improper priors commonly used for estimation. It allows us to compute the Bayes factor to compare the null and the alternative hypotheses. This default procedure of model selection is compared with a frequentist test and the Bayesian information criterion. We find discrepancy in the sense that frequentist and Bayesian information criterion reject the null hypothesis for data, that the Bayes factor for intrinsic or fractional priors do not.     

A Bayesian mixture model for differential gene expression

Summary.  We propose model-based inference for differential gene expression, using a nonparametric Bayesian probability model for the distribution of gene intensities under various conditions. The probability model is a mixture of normal distributions. The resulting inference is similar to a popular empirical Bayes approach that is used for the same inference problem. The use of fully model-based inference mitigates some of the necessary limitations of the empirical Bayes method. We argue that inference is no more difficult than posterior simulation in traditional nonparametric mixture-of-normal models. The approach proposed is motivated by a microarray experiment that was carried out to identify genes that are differentially expressed between normal tissue and colon cancer tissue samples. Additionally, we carried out a small simulation study to verify the methods proposed. In the motivating case-studies we show how the nonparametric Bayes approach facilitates the evaluation of posterior expected false discovery rates. We also show how inference can proceed even in the absence of a null sample of known non-differentially expressed scores. This highlights the difference from alternative empirical Bayes approaches that are based on plug-in estimates.     

Intrinsic Priors for Model Selection Using an Encompassing Model with Applications to Censored Failure Time Data

In Bayesian model selection or testingproblems one cannot utilize standard or default noninformativepriors, since these priors are typically improper and are definedonly up to arbitrary constants. Therefore, Bayes factors andposterior probabilities are not well defined under these noninformativepriors, making Bayesian model selection and testing problemsimpossible. We derive the intrinsic Bayes factor (IBF) of Bergerand Pericchi (1996a, 1996b) for the commonly used models in reliabilityand survival analysis using an encompassing model. We also deriveproper intrinsic priors for these models, whose Bayes factors are asymptoticallyequivalent to the respective IBFs. We demonstrate our resultsin three examples.     

Assessing the contribution of R&;D to total factor productivity—a Bayesian approach to account for heterogeneity and heteroskedasticity

This paper proposes a hierarchical Bayes estimator for a panel data random coefficient model with heteroskedasticity to assess the contribution of R&D capital to total factor productivity. Based on Hall (1993) data for 323 US firms over 1976–1990, we find that there appear to have substantial unobserved heterogeneity and heteroskedasticity across firms and industries that support the use of our Bayes inference procedure. We find much higher returns to R&D capital and a more pronounced downswing for the 1981–1985 period, followed by a more pronounced upswing than those yielded by the conventional feasible generalized least squares estimators or other estimates. The estimated elasticities of R&D capital are 0.062 for 1976–1980, 0.036 for 1981–1985 and 0.081 for 1986–1990, while the estimated elasticities of ordinary capital are much more stable over these periods.     

Objective Bayesian multiple comparisons for normal variances

This paper considers the multiple comparisons problem for normal variances. We propose a solution based on a Bayesian model selection procedure to this problem in which no subjective input is considered. We construct the intrinsic and fractional priors for which the Bayes factors and model selection probabilities are well defined. The posterior probability of each model is used as a model selection tool. The behaviour of these Bayes factors is compared with the Bayesian information criterion of Schwarz and some frequentist tests.     

Calibrated Bayes factors for model comparison

A Bayes factor between two models can be greatly affected by the prior distributions on the model parameters. When prior information is weak, very dispersed proper prior distributions are known to create a problem for the Bayes factor when competing models differ in dimension, and it is of even greater concern when one of the models is of infinite dimension. Therefore, we propose an innovative method which uses training samples to calibrate the prior distributions so that they achieve a reasonable level of ‘information’. Then the calibrated Bayes factor can be computed over the remaining data. This method makes no assumption on model forms (parametric or nonparametric) and can be used with both proper and improper priors. We illustrate, through simulation studies and a real data example, that the calibrated Bayes factor yields robust and reliable model preferences under various situations.     

On the wavelet-based simulation of anomalous diffusion

The field of microrheology is based on experiments involving particle diffusion. Microscopic tracer beads are placed into a non-Newtonian fluid and tracked using high speed video capture and light microscopy. The modelling of the behaviour of these beads is now an active scientific area which demands multiple stochastic and statistical methods. We propose an approximate wavelet-based simulation technique for two classes of continuous time anomalous diffusion models, the fractional Ornstein–Uhlenbeck process and the fractional generalized Langevin equation. The proposed algorithm is an iterative method that provides approximate discretizations that converge quickly and in an appropriate sense to the continuous time target process. As compared to previous works, it covers cases where the natural discretization of the target process does not have closed form in the time domain. Moreover, we propose to minimize the border effect via smoothing.     

A new class of minimax generalized Bayes estimators of a normal variance

A new class of minimax generalized Bayes estimators of the variance of a normal distribution is given under both quadratic and entropy losses. One contribution of the paper is a new class of minimax generalized Bayes estimators of a particularly simple form. Another contribution is a class of minimax generalized Bayes procedures satisfying a Strawderman [1974. Minimax estimation of powers of the variance of a normal population under squared error loss. Ann. Statist. 2, 190–198]-type condition which do not satisfy a Brewster and Zidek [1974. Improving on equivariant estimators. Ann. Statist. 2, 21–38]-type condition. We indicate that the new class may have a noticeably larger region of substantial improvement over the usual estimator than Brewster and Zidek-type procedures.     

Sensitivity of the fractional Bayes factor to prior distributions

The authors derive a measure of the sensitivity of the fractional Bayes factor, an index which is used to compare models when the priors for their respective parameters are improper, or when there is concern about robustness of the prior specification. They prove that in a large class of problems, this measure is a decreasing function of the fraction of the sample used to update the prior distribution before the models are compared.