A random weighting approach for posterior distributions

ABSTRACT

In Bayesian theory, calculating a posterior probability distribution is highly important but typically difficult. Therefore, some methods have been proposed to deal with such problem, among which, the most popular one is the asymptotic expansions of posterior distributions. In this paper, we propose an alternative approach, named a random weighting method, for scaled posterior distributions, and give an ideal convergence rate, o(n^{( ? 1/2)}), which serves as the theoretical guarantee for methods of numerical simulations.     

Data reconciliation of nonnormal observations with nonlinear constraints

This paper presents a new method for the reconciliation of data described by arbitrary continuous probability distributions, with the focus on nonlinear constraints. The main idea, already applied to linear constraints in a previous paper, is to restrict the joint prior probability distribution of the observed variables with model constraints to get a joint posterior probability distribution. Because in general the posterior probability density function cannot be calculated analytically, it is shown that it has decisive advantages to sample from the posterior distribution by a Markov chain Monte Carlo (MCMC) method. From the resulting sample of observed and unobserved variables various characteristics of the posterior distribution can be estimated, such as the mean, the full covariance matrix, marginal posterior densities, as well as marginal moments, quantiles, and HPD intervals. The procedure is illustrated by examples from material flow analysis and chemical engineering.     

Testing equality of regression coefficients in heteroscedastic normal regression models

This article addresses the problem of testing whether the vectors of regression coefficients are equal for two independent normal regression models when the error variances are unknown. This problem poses severe difficulties both to the frequentist and Bayesian approaches to statistical inference. In the former approach, normal hypothesis testing theory does not apply because of the unrelated variances. In the latter, the prior distributions typically used for the parameters are improper and hence the Bayes factor-based solution cannot be used.We propose a Bayesian solution to this problem in which no subjective input is considered. We first generate “objective” proper prior distributions (intrinsic priors) for which the Bayes factor and model posterior probabilities are well defined. The posterior probability of each model is used as a model selection tool. This consistent procedure of testing hypotheses is compared with some of the frequentist approximate tests proposed in the literature.     

Bayesian kernel projections for classification of high dimensional data

A Bayesian multi-category kernel classification method is proposed. The algorithm performs the classification of the projections of the data to the principal axes of the feature space. The advantage of this approach is that the regression coefficients are identifiable and sparse, leading to large computational savings and improved classification performance. The degree of sparsity is regulated in a novel framework based on Bayesian decision theory. The Gibbs sampler is implemented to find the posterior distributions of the parameters, thus probability distributions of prediction can be obtained for new data points, which gives a more complete picture of classification. The algorithm is aimed at high dimensional data sets where the dimension of measurements exceeds the number of observations. The applications considered in this paper are microarray, image processing and near-infrared spectroscopy data.     

On a theorem of Stein relating Bayesian and classical inferences in group models

Let a group G act on the sample space. This paper gives another proof of a theorem of Stein relating a group invariant family of posterior Bayesian probability regions to classical confidence regions when an appropriate prior is used. The example of the central multivariate normal distribution is discussed.     

Rubbery Polya Tree

Abstract. Polya trees (PT) are random probability measures which can assign probability 1 to the set of continuous distributions for certain specifications of the hyperparameters. This feature distinguishes the PT from the popular Dirichlet process (DP) model which assigns probability 1 to the set of discrete distributions. However, the PT is not nearly as widely used as the DP prior. Probably the main reason is an awkward dependence of posterior inference on the choice of the partitioning subsets in the definition of the PT. We propose a generalization of the PT prior that mitigates this undesirable dependence on the partition structure, by allowing the branching probabilities to be dependent within the same level. The proposed new process is not a PT anymore. However, it is still a tail‐free process and many of the prior properties remain the same as those for the PT.     

Kernel Likelihood Inference for Time Series

Abstract.  This paper develops non-parametric techniques for dynamic models whose data have unknown probability distributions. Point estimators are obtained from the maximization of a semiparametric likelihood function built on the kernel density of the disturbances. This approach can also provide Kullback–Leibler cross-validation estimates of the bandwidth of the kernel densities. Confidence regions are derived from the dual-empirical likelihood method based on non-parametric estimates of the scores. Limit theorems for martingale difference sequences support the statistical theory; moreover, simulation experiments and a real case study show the validity of the methods.     

Bayesian Analysis of Contingency Tables

ABSTRACT

The display of the data by means of contingency tables is used in different approaches to statistical inference, for example, to broach the test of homogeneity of independent multinomial distributions. We develop a Bayesian procedure to test simple null hypotheses versus bilateral alternatives in contingency tables. Given independent samples of two binomial distributions and taking a mixed prior distribution, we calculate the posterior probability that the proportion of successes in the first population is the same as in the second. This posterior probability is compared with the p-value of the classical method, obtaining a reconciliation between both results, classical and Bayesian. The obtained results are generalized for r × s tables.     

LOGISTIC REGRESSION COMPARED TO NORMAL DISCRIMINATION FOR NON-NORMAL POPULATIONS'

An investigation is undertaken of the logistic regression procedure for estimating the posterior probability of an object belonging to one of two populations. The asymptotic bias and mean square error associated with the procedure are derived for univariate populations whose distributions satisfy the general Day-Kerridge model for which the logistic form is valid for the posterior probability. These properties are compared with those of the normal discrimination method based on the classical assumption of normal populations with common variances. The asymptotic relative efficiency of logistic regression is considered on the basis of asymptotic mean square error.     

A unified approach to minimum variance unbiased estimation of a probability function belonging to an exponential family

The problem of minimum variance unbiased estimation of the probability density function of a random variable belonging to an exponential family is considered. The method of estimation proposed in this paper requires the solution of a certain integral equation. For many probability distributions the solution of this equation is given by a known result in integral transform theory.     

Coherent Frequentism: A Decision Theory Based on Confidence Sets

By representing fair betting odds according to one or more pairs of confidence set estimators, dual parameter distributions called confidence posteriors secure the coherence of actions without any prior distribution. This theory reduces to the maximization of expected utility when the pair of posteriors is induced by an exact or approximate confidence set estimator or when a reduction rule is applied to the pair. Unlike the p-value, the confidence posterior probability of an interval hypothesis is suitable as an estimator of the indicator of hypothesis truth since it converges to 1 if the hypothesis is true or to 0 otherwise.     

Distribution of the product of a pair of independent two-sided power variates

ABSTRACT

The distributions of algebraic functions of random variables are important in theory of probability and statistics and other areas such as engineering, reliability, and actuarial applications, and many results based on various distributions are available in the literature. The two-sided power distribution is defined on a bounded range, and it is a generalization of the uniform, triangular, and power-function probability distributions. This paper gives the exact distribution of the product of two independent two-sided power-distributed random variables in a computable representation. The percentiles of the product are then computed, and a real data application is given.     

A Bayesian growth and yield model for slash pine plantations

We formulate a traditional growth and yield model as a Bayes model. We attempt to introduce as few new assumptions as possible. Zellner's Bayesian method of moments procedure is used, because the published model did not include any distributional assumptions. We generate predictive posterior samples for a number of stand variables using the Gibbs sampler. The means of the samples compare favorably with the predictions from the published model. In addition, our model delivers distributions of outcomes, from which it is easy to establish measures of uncertainty, such as highest posterior density regions.     

Empirical Bayes Confidence Intervals for Means of Natural Exponential Family-Quadratic Variance Function Distributions with Application to Small Area Estimation

Abstract.  The paper develops empirical Bayes (EB) confidence intervals for population means with distributions belonging to the natural exponential family-quadratic variance function (NEF-QVF) family when the sample size for a particular population is moderate or large. The basis for such development is to find an interval centred around the posterior mean which meets the target coverage probability asymptotically, and then show that the difference between the coverage probabilities of the Bayes and EB intervals is negligible up to a certain order. The approach taken is Edgeworth expansion so that the sample sizes from the different populations need not be significantly large. The proposed intervals meet the target coverage probabilities asymptotically, and are easy to construct. We illustrate use of these intervals in the context of small area estimation both through real and simulated data. The proposed intervals are different from the bootstrap intervals. The latter can be applied quite generally, but the order of accuracy of these intervals in meeting the desired coverage probability is unknown.     

Multiple testing using the posterior probabilities of directional alternatives,with application to genomic studies

This paper aims to connect Bayesian analysis and frequentist theory in the context of multiple comparisons. The authors show that when testing the equality of two sample means, the posterior probability of the one‐sided alternative hypothesis, defined as a half‐space, shares with the frequentist P‐value the property of uniformity under the null hypothesis. Ultimately, the posterior probability may thus be used in the same spirit as a P‐value in the Benjamini‐Hochberg procedure, or in any of its extensions.     

Irreconcilability of P-value and Bayesian measure in two-sided hypothesis: Pareto distribution with the presence of nuisance parameter

Abstract

This article is concerned with the comparison of Bayesian and classical testing of a point null hypothesis for the Pareto distribution when there is a nuisance parameter. In the first stage, using a fixed prior distribution, the posterior probability is obtained and compared with the P-value. In the second case, lower bounds of the posterior probability of H₀, under a reasonable class of prior distributions, are compared with the P-value. It has been shown that even in the presence of nuisance parameters for the model, these two approaches can lead to different results in statistical inference.     

A Bayesian growth and yield model for slash pine plantations

We formulate a traditional growth and yield model as a Bayes model. We attempt to introduce as few new assumptions as possible. Zellner's Bayesian method of moments procedure is used, because the published model did not include any distributional assumptions. We generate predictive posterior samples for a number of stand variables using the Gibbs sampler. The means of the samples compare favorably with the predictions from the published model. In addition, our model delivers distributions of outcomes, from which it is easy to establish measures of uncertainty, such as highest posterior density regions.     

Teaching Bayes’ Theorem: Strength of Evidence as Predictive Accuracy

Although teaching Bayes’ theorem is popular, the standard approach—targeting posterior distributions of parameters—may be improved. We advocate teaching Bayes’ theorem in a ratio form where the posterior beliefs relative to the prior beliefs equals the conditional probability of data relative to the marginal probability of data. This form leads to an interpretation that the strength of evidence is relative predictive accuracy. With this approach, students are encouraged to view Bayes’ theorem as an updating mechanism, to obtain a deeper appreciation of the role of the prior and of marginal data, and to view estimation and model comparison from a unified perspective.     

Exact Bayesian modeling for bivariate Poisson data and extensions

Bivariate count data arise in several different disciplines (epidemiology, marketing, sports statistics just to name a few) and the bivariate Poisson distribution being a generalization of the Poisson distribution plays an important role in modelling such data. In the present paper we present a Bayesian estimation approach for the parameters of the bivariate Poisson model and provide the posterior distributions in closed forms. It is shown that the joint posterior distributions are finite mixtures of conditionally independent gamma distributions for which their full form can be easily deduced by a recursively updating scheme. Thus, the need of applying computationally demanding MCMC schemes for Bayesian inference in such models will be removed, since direct sampling from the posterior will become available, even in cases where the posterior distribution of functions of the parameters is not available in closed form. In addition, we define a class of prior distributions that possess an interesting conjugacy property which extends the typical notion of conjugacy, in the sense that both prior and posteriors belong to the same family of finite mixture models but with different number of components. Extension to certain other models including multivariate models or models with other marginal distributions are discussed.     

Some useful integrals and their applications in correlation analysis

The distribution of the product moment correlation coefficient based on the bivariate normal distribution is well known. Recently in many business and economic data, fat tailed distributions especially some elliptical distributions have been considered as parent populations. The normal and t-distributions are well known special cases of elliptical distribution. In this paper we derive some theorems involving double integrals and apply them to derive the probability distribution of the correlation coefficient for some elliptical populations. The general nature of the theorems indicates their potential use in probability distribution theory.