Eliciting Dirichlet and Gaussian copula prior distributions for multinomial models

In this paper, we propose novel methods of quantifying expert opinion about prior distributions for multinomial models. Two different multivariate priors are elicited using median and quartile assessments of the multinomial probabilities. First, we start by eliciting a univariate beta distribution for the probability of each category. Then we elicit the hyperparameters of the Dirichlet distribution, as a tractable conjugate prior, from those of the univariate betas through various forms of reconciliation using least-squares techniques. However, a multivariate copula function will give a more flexible correlation structure between multinomial parameters if it is used as their multivariate prior distribution. So, second, we use beta marginal distributions to construct a Gaussian copula as a multivariate normal distribution function that binds these marginals and expresses the dependence structure between them. The proposed method elicits a positive-definite correlation matrix of this Gaussian copula. The two proposed methods are designed to be used through interactive graphical software written in Java.     

Bayesian inference for Poisson and multinomial log-linear models

Categorical data frequently arise in applications in the Social Sciences. In such applications, the class of log-linear models, based on either a Poisson or (product) multinomial response distribution, is a flexible model class for inference and prediction. In this paper we consider the Bayesian analysis of both Poisson and multinomial log-linear models. It is often convenient to model multinomial or product multinomial data as observations of independent Poisson variables. For multinomial data, Lindley (1964) [20] showed that this approach leads to valid Bayesian posterior inferences when the prior density for the Poisson cell means factorises in a particular way. We develop this result to provide a general framework for the analysis of multinomial or product multinomial data using a Poisson log-linear model. Valid finite population inferences are also available, which can be particularly important in modelling social data. We then focus particular attention on multivariate normal prior distributions for the log-linear model parameters. Here, an improper prior distribution for certain Poisson model parameters is required for valid multinomial analysis, and we derive conditions under which the resulting posterior distribution is proper. We also consider the construction of prior distributions across models, and for model parameters, when uncertainty exists about the appropriate form of the model. We present classes of Poisson and multinomial models, invariant under certain natural groups of permutations of the cells. We demonstrate that, if prior belief concerning the model parameters is also invariant, as is the case in a ‘reference’ analysis, then the choice of prior distribution is considerably restricted. The analysis of multivariate categorical data in the form of a contingency table is considered in detail. We illustrate the methods with two examples.     

Bayesian identifiability and misclassification in multinomial data

The authors consider the Bayesian analysis of multinomial data in the presence of misclassification. Misclassification of the multinomial cell entries leads to problems of identifiability which are categorized into two types. The first type, referred to as the permutation‐type nonidentifiabilities, may be handled with constraints that are suggested by the structure of the problem. Problems of identifiability of the second type are addressed with informative prior information via Dirichlet distributions. Computations are carried out using a Gibbs sampling algorithm.     

A bayesian analysis of the multinomial model for a dichotomous response with nonrespondents

This paper studies the mu1tinomial model 2x2 contingency table data with some cell counts missing .Various hypotheses of interest including row-column independence are tested by using Bayes factors which represent the ratio of the posterior odds to the prior odds for the null hypothesis. The Dirichlet-Beta family of prior distributions is considered for the multinomial parameters cond itional on the complement of the null hypothesis. The Bayes factor for the incomplete data is a mixture of the Bayes factors for different possibilities for the full data.     

An extension of the Dirichlet prior for the analysis of longitudinal multinomial data

Studies producing longitudinal multinomial data arise in several subject areas. This article suggests a Bayesian approach to the analysis of such data. Rather than infusing a latent model structure, we develop a prior distribution for the multinomial parameters which reflects the longitudinal nature of the observations. This distribution is constructed by modifying the prior that posits independent Dirichlet distributions for the multinomial parameters across time. Posterior analysis, which is implemented using Monte Carlo methods, can then be used to assess the temporal behaviour of the multinomial parameters underlying the observed data. We test this methodology on simulated data, opinion polling data, and data from a study concerning the development of moral reasoning.     

Some properties of the dirichlet-multinomial distribution and its use in prior elicitation

Two results on the unimodality of the Dirichlet-multinomial distribution are proved, and a further result is alos proved on the identifiability of mixtures of multinomial distributions. These properties are used in developing a method for eliciting a Dirchlet prior distribution. The elicitation method is based on the mode, and region around the mode, of the Dirichlet-multinomial predictive distribution.     

Posterior Distributions for the Gini Coefficient Using Grouped Data

When available data comprise a number of sampled households in each of a number of income classes, the likelihood function is obtained from a multinomial distribution with the income class population proportions as the unknown parameters. Two methods for going from this likelihood function to a posterior distribution on the Gini coefficient are investigated. In the first method, two alternative assumptions about the underlying income distribution are considered, namely a lognormal distribution and the Singh–Maddala (1976) income distribution. In these cases the likelihood function is reparameterized and the Gini coefficient is a nonlinear function of the income distribution parameters. The Metropolis algorithm is used to find the corresponding posterior distributions of the Gini coefficient from a sample of Bangkok households. The second method does not require an assumption about the nature of the income distribution, but uses (a) triangular prior distributions, and (b) beta prior distributions, on the location of mean income within each income class. By sampling from these distributions, and the Dirichlet posterior distribution of the income class proportions, alternative posterior distributions of the Gini coefficient are calculated.     

Data augmentation,frequentist estimation,and the Bayesian analysis of multinomial logit models

This article describes a convenient method of selecting Metropolis– Hastings proposal distributions for multinomial logit models. There are two key ideas involved. The first is that multinomial logit models have a latent variable representation similar to that exploited by Albert and Chib (J Am Stat Assoc 88:669–679, 1993) for probit regression. Augmenting the latent variables replaces the multinomial logit likelihood function with the complete data likelihood for a linear model with extreme value errors. While no conjugate prior is available for this model, a least squares estimate of the parameters is easily obtained. The asymptotic sampling distribution of the least squares estimate is Gaussian with known variance. The second key idea in this paper is to generate a Metropolis–Hastings proposal distribution by conditioning on the estimator instead of the full data set. The resulting sampler has many of the benefits of so-called tailored or approximation Metropolis–Hastings samplers. However, because the proposal distributions are available in closed form they can be implemented without numerical methods for exploring the posterior distribution. The algorithm converges geometrically ergodically, its computational burden is minor, and it requires minimal user input. Improvements to the sampler’s mixing rate are investigated. The algorithm is also applied to partial credit models describing ordinal item response data from the 1998 National Assessment of Educational Progress. Its application to hierarchical models and Poisson regression are briefly discussed.     

On binomial and circular binomial distributions of orderk forl-overlapping success runs of lengthk

The number ofl-overlapping success runs of lengthk inn trials, which was introduced and studied recently, is presently reconsidered in the Bernoulli case and two exact formulas are derived for its probability distribution function in terms of multinomial and binomial coefficients respectively. A recurrence relation concerning this distribution, as well as its mean, is also obtained. Furthermore, the number ofl-overlapping success runs of lengthk inn Bernoulli trials arranged on a circle is presently considered for the first time and its probability distribution function and mean are derived. Finally, the latter distribution is related to the first, two open problems regarding limiting distributions are stated, and numerical illustrations are given in two tables. All results are new and they unify and extend several results of various authors on binomial and circular binomial distributions of orderk.     

Bayesian Analysis of Contingency Tables

ABSTRACT

The display of the data by means of contingency tables is used in different approaches to statistical inference, for example, to broach the test of homogeneity of independent multinomial distributions. We develop a Bayesian procedure to test simple null hypotheses versus bilateral alternatives in contingency tables. Given independent samples of two binomial distributions and taking a mixed prior distribution, we calculate the posterior probability that the proportion of successes in the first population is the same as in the second. This posterior probability is compared with the p-value of the classical method, obtaining a reconciliation between both results, classical and Bayesian. The obtained results are generalized for r × s tables.     

The application of robust Bayesian analysis to hypothesis testing and Occam's Razor

Summary Robust Bayesian analysis deals simultaneously with a class of possible prior distributions, instead of a single distribution. This paper concentrates on the surprising results that can be obtained when applying the theory to problems of testing precise hypotheses when the “objective” class of prior distributions is assumed. First, an example is given demonstrating the serious inadequacy of P-values for this problem. Next, it is shown how the approach can provide statistical quantification of Occam's Razor, the famous principle of science that advocates choice of the simpler of two hypothetical explanations of data. Finally, the theory is applied to multinomial testing. Research supported by the National Science Foundation, Grant DMS-8923071, and by NASA Contract NAS5-29285 for the hubble Space Telescope.     

Divergence statistics: sampling properties and multinomial goodness of fit and divergence tests

φ-divergence .statistics are obtained by either replacing both distributions involved in the argument of the φ -divergence measure by their sample estimates or replacing one distribution and considering the other as given. The sampling properties of estimated divergence-type measures are investigated. Approximate means and variances are derived and asymptotic distributions are obtained. Tests of goodness of fit of observed frequencies to expected ones and tests of equality of divergences based on two or more multinomial samples are constructed.     

Prbposterior expected loss as a scoring rule for prior distributions

A scoring rule for evaluating the usefulness of an assessed prior distribution should reflect the purpose for which the distribution is to be used. In this paper we suppose that sample data is to become available and that the posterior distribution will be used to estimate some quantity under a quadratic loss function. The utility of a prior distribution is consequently determined by its preposterior expected quadratic loss. It is shown that this loss function has properties desirable in a scoring rule and formulae are derived for calculating the scores it gives in some common problems. Many scoring rules give a very poor score to any improper prior distribution but, in contrast, the scoring rule proposed here provides a meaningful measure for comparing the usefulness of assessed prior distributions and non-informative (improper) prior distributions. Results for making this comparison in various situations are also given.     

Consistent Estimation of the Structural Distribution Function

Motivated by problems in linguistics we consider a multinomial random vector for which the number of cells N is not much smaller than the sum of the cell frequencies, i.e. the sample size n . The distribution function of the uniform distribution on the set of all cell probabilities multiplied by N is called the structural distribution function of the cell probabilities. Conditions are given that guarantee that the structural distribution function can be estimated consistently as n increases indefinitely although n / N does not. The natural estimator is inconsistent and we prove consistency of essentially two alternative estimators.     

Bayesian methods for generalized linear models with covariates missing at random

The authors propose methods for Bayesian inference for generalized linear models with missing covariate data. They specify a parametric distribution for the covariates that is written as a sequence of one‐dimensional conditional distributions. They propose an informative class of joint prior distributions for the regression coefficients and the parameters arising from the covariate distributions. They examine the properties of the proposed prior and resulting posterior distributions. They also present a Bayesian criterion for comparing various models, and a calibration is derived for it. A detailed simulation is conducted and two real data sets are examined to demonstrate the methodology.     

On robust Bayesian estimation under some asymmetric and bounded loss function

The problem of Bayesian and robust Bayesian estimation with some bounded and asymmetric loss function ABL is considered for various models. The prior distribution is not exactly specified and covers the conjugate family of prior distributions. The posterior regret, most robust and conditional Γ-minimax estimators are constructed and a preliminary comparison with square-error loss and LINEX loss is presented.     

Large sample results for tests of association ii. multinomial and stratified sampling

This paper deals with the asymptotics of a class of tests for association in 2-way contingency tables based on square forms in cell frequencies, given the total number of observations (multinomial sampling) or one set of marginal totals (stratified sampling). The case when both row and column marginal totals are fixed (hypergeometric sampling) was studied in Kulinskaya (1994), The class of tests under consideration includes a number of classical measures for association, Its two subclasses are the tests based on statistics using centralized cell frequencies (asymptotically distributed as weighted sums of central chi-squares) and those using the non-centralized cell frequencies (asymptotically normal). The parameters of asymptotic distributions depend on the sampling model and on true marginal probabilities. Maximum efficiency for asymptotically normal statistics is achieved under hypergeometric sampling, If the cell frequencies or the statistic as a whole are centralized using marginal proportions as estimates for marginal probabilities, the asymptotic distribution does not differ much between models and it is equivalent to that under hypergeometric sampling. These findings give an extra justification for the use of permutation tests for association (which are based on hypergeometric sampling). As an application, several well known measures of association are analysed.     

Joint distribution of simultaneous exposures to several carcinogens in a case–control study: sample size determination

Consider a population of individuals who are free of a disease under study, and who are exposed simultaneously at random exposure levels, say X,Y,Z,… to several risk factors which are suspected to cause the disease in the populationm. At any specified levels X=x, Y=y, Z=z, …, the incidence rate of the disease in the population ot risk is given by the exposure–response relationship r(x,y,z,…) = P(disease|x,y,z,…). The present paper examines the relationship between the joint distribution of the exposure variables X,Y,Z, … in the population at risk and the joint distribution of the exposure variables U,V,W,… among cases under the linear and the exponential risk models. It is proven that under the exponential risk model, these two joint distributions belong to the same family of multivariate probability distributions, possibly with different parameters values. For example, if the exposure variables in the population at risk have jointly a multivariate normal distribution, so do the exposure variables among cases; if the former variables have jointly a multinomial distribution, so do the latter. More generally, it is demonstrated that if the joint distribution of the exposure variables in the population at risk belongs to the exponential family of multivariate probability distributions, so does the joint distribution of exposure variables among cases. If the epidemiologist can specify the differnce among the mean exposure levels in the case and control groups which are considered to be clinically or etiologically important in the study, the results of the present paper may be used to make sample size determinations for the case–control study, corresponding to specified protection levels, i.e., size α and 1–β of a statistical test. The multivariate normal, the multinomial, the negative multinomial and Fisher's multivariate logarithmic series exposure distributions are used to illustrate our results.     

Modelling Heterogeneity With and Without the Dirichlet Process

We investigate the relationships between Dirichlet process (DP) based models and allocation models for a variable number of components, based on exchangeable distributions. It is shown that the DP partition distribution is a limiting case of a Dirichlet–multinomial allocation model. Comparisons of posterior performance of DP and allocation models are made in the Bayesian paradigm and illustrated in the context of univariate mixture models. It is shown in particular that the unbalancedness of the allocation distribution, present in the prior DP model, persists a posteriori . Exploiting the model connections, a new MCMC sampler for general DP based models is introduced, which uses split/merge moves in a reversible jump framework. Performance of this new sampler relative to that of some traditional samplers for DP processes is then explored.     

Analysis of over-dispersed count data with extra zeros using the Poisson log-skew-normal distribution

ABSTRACT

Mixed Poisson distributions are widely used in various applications of count data mainly when extra variation is present. This paper introduces an extension in terms of a mixed strategy to jointly deal with extra-Poisson variation and zero-inflated counts. In particular, we propose the Poisson log-skew-normal distribution which utilizes the log-skew-normal as a mixing prior and present its main properties. This is directly done through additional hierarchy level to the lognormal prior and includes the Poisson lognormal distribution as its special case. Two numerical methods are developed for the evaluation of associated likelihoods based on the Gauss–Hermite quadrature and the Lambert's W function. By conducting simulation studies, we show that the proposed distribution performs better than several commonly used distributions that allow for over-dispersion or zero inflation. The usefulness of the proposed distribution in empirical work is highlighted by the analysis of a real data set taken from health economics contexts.