Bayesian revision of a prior given prior-data conflict,expert opinion,or a similar insight: a large-deviation approach

Learning from model diagnostics that a prior distribution must be replaced by one that conflicts less with the data raises the question of which prior should instead be used for inference and decision. The same problem arises when a decision maker learns that one or more reliable experts express unexpected beliefs. In both cases, coherence of the solution would be guaranteed by applying Bayes's theorem to a distribution of prior distributions that effectively assigns the initial prior distribution a probability arbitrarily close to 1. The new distribution for inference would then be the distribution of priors conditional on the insight that the prior distribution lies in a closed convex set that does not contain the initial prior. A readily available distribution of priors needed for such conditioning is the law of the empirical distribution of sufficiently large number of independent parameter values drawn from the initial prior. According to the Gibbs conditioning principle from the theory of large deviations, the resulting new prior distribution minimizes the entropy relative to the initial prior. While minimizing relative entropy accommodates the necessity of going beyond the initial prior without departing from it any more than the insight demands, the large-deviation derivation also ensures the advantages of Bayesian coherence. This approach is generalized to uncertain insights by allowing the closed convex set of priors to be random.     

On independence of Markov kernels and a generalization of two theorems of Basu

Markov kernels play an important role in probability theory and mathematical statistics, conditional distributions being the main example.     

A Gaussian alternative to using improper confidence intervals

The problem posed by exact confidence intervals (CIs) which can be either all-inclusive or empty for a nonnegligible set of sample points is known to have no solution within CI theory. Confidence belts causing improper CIs can be modified by using margins of error from the renewed theory of errors initiated by J. W. Tukey—briefly described in the article—for which an extended Fraser's frequency interpretation is given. This approach is consistent with Kolmogorov's axiomatization of probability, in which a probability and an error measure obey the same axioms, although the connotation of the two words is different. An algorithm capable of producing a margin of error for any parameter derived from the five parameters of the bivariate normal distribution is provided. Margins of error correcting Fieller's CIs for a ratio of means are obtained, as are margins of error replacing Jolicoeur's CIs for the slope of the major axis. Margins of error using Dempster's conditioning that can correct optimal, but improper, CIs for the noncentrality parameter of a noncentral chi-square distribution are also given.     

Observations on the effect of the prior distribution on the predictive distribution in Bayesian inferences

Typically, in the brief discussion of Bayesian inferential methods presented at the beginning of calculus-based undergraduate or graduate mathematical statistics courses, little attention is paid to the process of choosing the parameter value(s) for the prior distribution. Even less attention is paid to the impact of these choices on the predictive distribution of the data. Reasons for this include that the posterior can be found by ignoring the predictive distribution thereby streamlining the derivation of the posterior and/or that computer software can be used to find the posterior distribution. In this paper, the binomial, negative-binomial and Poisson distributions along with their conjugate beta and gamma priors are utilized to obtain the resulting predictive distributions. It is then demonstrated that specific choices of the parameters of the priors can lead to predictive distributions with properties that might be surprising to a non-expert user of Bayesian methods.     

Bayesian analysis of elapsed times in continuous‐time Markov chains

The authors consider Bayesian analysis for continuous‐time Markov chain models based on a conditional reference prior. For such models, inference of the elapsed time between chain observations depends heavily on the rate of decay of the prior as the elapsed time increases. Moreover, improper priors on the elapsed time may lead to improper posterior distributions. In addition, an infinitesimal rate matrix also characterizes this class of models. Experts often have good prior knowledge about the parameters of this matrix. The authors show that the use of a proper prior for the rate matrix parameters together with the conditional reference prior for the elapsed time yields a proper posterior distribution. The authors also demonstrate that, when compared to analyses based on priors previously proposed in the literature, a Bayesian analysis on the elapsed time based on the conditional reference prior possesses better frequentist properties. The type of prior thus represents a better default prior choice for estimation software.     

A limit theorem on moment convergence of centered spectral statistics of random matrices

ABSTRACT

The eigenvalues of a random matrix are a sequence of specific dependent random variables, the limiting properties of which are one of interesting topics in probability theory. The aim of the article is to extend some probability limiting properties of i.i.d. random variables in the context of the complete moment convergence to the centered spectral statistics of random matrices. Some precise asymptotic results related to the complete convergence of p-order conditional moment of Wigner matrices and sample covariance matrices are obtained. The proofs mainly depend on the central limit theorem and large deviation inequalities of spectral statistics.     

Detecting parameter shift in garch models

This paper applies recent theories of testing for parameter constancy to the conditional variance in a GARCH model. The supremum Lagrange multiplier test for conditional Gaussian GARCH models and its robustified variants are discussed. The asymptotic null distribution of the test statistics are derived from the weak convergence of the scores, and the critical values from the hitting probability of squared Bessel process.

Monte Carlo studies on the finite sample size and power performance of the supremum LM tests are conducted. Applications of these tests to S&P 500 indicate that the hypothesis of stable conditional variance parameters can be rejected.     

Weak and strong laws of large numbers for coherent lower previsions

We prove weak and strong laws of large numbers for coherent lower previsions, where the lower prevision of a random variable is given a behavioural interpretation as a subject's supremum acceptable price for buying it. Our laws are a consequence of the rationality criterion of coherence, and they can be proven under assumptions that are surprisingly weak when compared to the standard formulation of the laws in more classical approaches to probability theory.     

Objective Bayesian analysis for a spatial model with nugget effects

The focus of this paper is objective priors for spatially correlated data with nugget effects. In addition to the Jeffreys priors and commonly used reference priors, two types of “exact” reference priors are derived based on improper marginal likelihoods. An “equivalence” theorem is developed in the sense that the expectation of any function of the score functions of the marginal likelihood function can be taken under marginal likelihoods. Interestingly, these two types of reference priors are identical.     

The mathematics of Benford’s law: a primer

This article provides a concise overview of the main mathematical theory of Benford’s law in a form accessible to scientists and students who have had first courses in calculus and probability. In particular, one of the main objectives here is to aid researchers who are interested in applying Benford’s law, and need to understand general principles clarifying when to expect the appearance of Benford’s law in real-life data and when not to expect it. A second main target audience is students of statistics or mathematics, at all levels, who are curious about the mathematics underlying this surprising and robust phenomenon, and may wish to delve more deeply into the subject. This survey of the fundamental principles behind Benford’s law includes many basic examples and theorems, but does not include the proofs or the most general statements of the theorems; rather it provides precise references where both may be found.

     

Asymptotic inference for mixture models by using data-dependent priors

For certain mixture models, improper priors are undesirable because they yield improper posteriors. However, proper priors may be undesirable because they require subjective input. We propose the use of specially chosen data-dependent priors. We show that, in some cases, data-dependent priors are the only priors that produce intervals with second-order correct frequentist coverage. The resulting posterior also has another interpretation: it is the product of a fixed prior and a pseudolikelihood.     

Estimation of entropy-type integral functionals

ABSTRACT

Entropy-type integral functionals of densities are widely used in mathematical statistics, information theory, and computer science. Examples include measures of closeness between distributions (e.g., density power divergence) and uncertainty characteristics for a random variable (e.g., Rényi entropy). In this paper, we study U-statistic estimators for a class of such functionals. The estimators are based on ε-close vector observations in the corresponding independent and identically distributed samples. We prove asymptotic properties of the estimators (consistency and asymptotic normality) under mild integrability and smoothness conditions for the densities. The results can be applied in diverse problems in mathematical statistics and computer science (e.g., distribution identification problems, approximate matching for random databases, two-sample problems).     

Can You Really Learn Basic Probability by Playing a Sports Board Game?

This article considers the use of sports board games to introduce or illustrate a wide variety of probability concepts to introductory statistics students in an integrated manner. We demonstrate the use of a single game (Strat-O-Matic® Baseball) to introduce probability distributions, sample spaces, the laws of addition and multiplication of probabilities, independence, mutual exclusivity, randomization and independence, conditional probability, and Bayes' Theorem. Empirical and anecdotal evidence suggests that student comprehension and retention are enhanced by use of examples constructed from the simple and interesting contexts provided by a sports board game.     

Intrinsic Priors for Testing Two Normal Means with Intrinsic Bayes Factors

ABSTRACT

In this article we consider the problem of comparing two normal means with unknown common variance using a Bayesian approach. Conventional Bayes factors with improper non informative priors are not well defined. The intrinsic Bayes factors are used to overcome such a difficulty. We derive intrinsic priors whose Bayes factors are asymptotically equivalent to the corresponding intrinsic Bayes factors. We illustrate our results with numerical examples.     

On a Class of Random Probability Measures with General Predictive Structure

Abstract. In this study, we investigate a recently introduced class of non‐parametric priors, termed generalized Dirichlet process priors. Such priors induce (exchangeable random) partitions that are characterized by a more elaborate clustering structure than those arising from other widely used priors. A natural area of application of these random probability measures is represented by species sampling problems and, in particular, prediction problems in genomics. To this end, we study both the distribution of the number of distinct species present in a sample and the distribution of the number of new species conditionally on an observed sample. We also provide the Bayesian Non‐parametric estimator for the number of new species in an additional sample of given size and for the discovery probability as function of the size of the additional sample. Finally, the study of its conditional structure is completed by the determination of the posterior distribution.     

Simulation-assisted saddlepoint approximation

A general saddlepoint/Monte Carlo method to approximate (conditional) multivariate probabilities is presented. This method requires a tractable joint moment generating function (m.g.f.), but does not require a tractable distribution or density. The method is easy to program and has a third-order accuracy with respect to increasing sample size in contrast to standard asymptotic approximations which are typically only accurate to the first order.

The method is most easily described in the context of a continuous regular exponential family. Here, inferences can be formulated as probabilities with respect to the joint density of the sufficient statistics or the conditional density of some sufficient statistics given the others. Analytical expressions for these densities are not generally available, and it is often not possible to simulate exactly from the conditional distributions to obtain a direct Monte Carlo approximation of the required integral. A solution to the first of these problems is to replace the intractable density by a highly accurate saddlepoint approximation. The second problem can be addressed via importance sampling, that is, an indirect Monte Carlo approximation involving simulation from a crude approximation to the true density. Asymptotic normality of the sufficient statistics suggests an obvious candidate for an importance distribution.

The more general problem considers the computation of a joint probability for a subvector of random T, given its complementary subvector, when its distribution is intractable, but its joint m.g.f. is computable. For such settings, the distribution may be tilted, maintaining T as the sufficient statistic. Within this tilted family, the computation of such multivariate probabilities proceeds as described for the exponential family setting.     

Intrinsic Bayes factor approach to a test for the power law process

The versatile new criterion called the intrinsic Bayes factor (IBF), introduced by Berger and Pericchi [J. Amer. Statist. Assoc. 91 (1996) 109–122], has made it possible to perform model selection and hypotheses testing using standard (improper) noninformative priors in a variety of situations. In this paper, we use their methodology to test several hypotheses regarding the shape parameter of the power law process, which has been widely used to model failure times of repairable systems. Assuming that we have data from the process according to the time-truncation sampling scheme, we derive the arithmetic IBFs using four default priors, including the reference and Jeffreys priors. We establish the frequentist probability matching properties of these priors. We also identify two priors that are justifiable under both time-truncation and failure-truncation schemes, so that the IBFs for both schemes can be unified. Deducing the intrinsic priors of a certain canonical form, as the time of truncation tends to infinity, we show that the arithmetic IBFs correspond asymptotically to actual Bayes factors. We also discuss the expected IBFs, which are useful with small samples. We then use these results to analyze an actual data set on the interruption times of a transmission line, summarizing our results under the default priors.     

Anna Karenina and the two envelopes problem

The Anna Karenina principle is named after the opening sentence in the eponymous novel: Happy families are all alike; every unhappy family is unhappy in its own way. The two envelopes problem (TEP) is a much-studied paradox in probability theory, mathematical economics, logic and philosophy. Time and again a new analysis is published in which an author claims finally to explain what actually goes wrong in this paradox. Each author (the present author included) emphasises what is new in their approach and concludes that earlier approaches did not get to the root of the matter. We observe that though a logical argument is only correct if every step is correct, an apparently logical argument which goes astray can be thought of as going astray at different places. This leads to a comparison between the literature on TEP and a successful movie franchise: it generates a succession of sequels, and even prequels, each with a different director who approaches the same basic premise in a personal way. We survey resolutions in the literature with a view to synthesis, correct common errors, and give a new theorem on order properties of an exchangeable pair of random variables, at the heart of most TEP variants and interpretations. A theorem on asymptotic independence between the amount in your envelope and the question whether it is smaller or larger shows that the pathological situation of improper priors or infinite expectation values has consequences as we merely approach such a situation.     

On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion)

New methodology for fully Bayesian mixture analysis is developed, making use of reversible jump Markov chain Monte Carlo methods that are capable of jumping between the parameter subspaces corresponding to different numbers of components in the mixture. A sample from the full joint distribution of all unknown variables is thereby generated, and this can be used as a basis for a thorough presentation of many aspects of the posterior distribution. The methodology is applied here to the analysis of univariate normal mixtures, using a hierarchical prior model that offers an approach to dealing with weak prior information while avoiding the mathematical pitfalls of using improper priors in the mixture context.     

Default Bayesian analysis of the Behrens–Fisher problem

In the Bayesian approach, the Behrens–Fisher problem has been posed as one of estimation for the difference of two means. No Bayesian solution to the Behrens–Fisher testing problem has yet been given due, perhaps, to the fact that the conventional priors used are improper. While default Bayesian analysis can be carried out for estimation purposes, it poses difficulties for testing problems. This paper generates sensible intrinsic and fractional prior distributions for the Behrens–Fisher testing problem from the improper priors commonly used for estimation. It allows us to compute the Bayes factor to compare the null and the alternative hypotheses. This default procedure of model selection is compared with a frequentist test and the Bayesian information criterion. We find discrepancy in the sense that frequentist and Bayesian information criterion reject the null hypothesis for data, that the Bayes factor for intrinsic or fractional priors do not.