Bayesian inference for Poisson and multinomial log-linear models

Categorical data frequently arise in applications in the Social Sciences. In such applications, the class of log-linear models, based on either a Poisson or (product) multinomial response distribution, is a flexible model class for inference and prediction. In this paper we consider the Bayesian analysis of both Poisson and multinomial log-linear models. It is often convenient to model multinomial or product multinomial data as observations of independent Poisson variables. For multinomial data, Lindley (1964) [20] showed that this approach leads to valid Bayesian posterior inferences when the prior density for the Poisson cell means factorises in a particular way. We develop this result to provide a general framework for the analysis of multinomial or product multinomial data using a Poisson log-linear model. Valid finite population inferences are also available, which can be particularly important in modelling social data. We then focus particular attention on multivariate normal prior distributions for the log-linear model parameters. Here, an improper prior distribution for certain Poisson model parameters is required for valid multinomial analysis, and we derive conditions under which the resulting posterior distribution is proper. We also consider the construction of prior distributions across models, and for model parameters, when uncertainty exists about the appropriate form of the model. We present classes of Poisson and multinomial models, invariant under certain natural groups of permutations of the cells. We demonstrate that, if prior belief concerning the model parameters is also invariant, as is the case in a ‘reference’ analysis, then the choice of prior distribution is considerably restricted. The analysis of multivariate categorical data in the form of a contingency table is considered in detail. We illustrate the methods with two examples.     

Data augmentation in multi-way contingency tables with fixed marginal totals

We describe and illustrate approaches to data augmentation in multi-way contingency tables for which partial information, in the form of subsets of marginal totals, is available. In such problems, interest lies in questions of inference about the parameters of models underlying the table together with imputation for the individual cell entries. We discuss questions of structure related to the implications for inference on cell counts arising from assumptions about log-linear model forms, and a class of simple and useful prior distributions on the parameters of log-linear models. We then discuss “local move” and “global move” Metropolis–Hastings simulation methods for exploring the posterior distributions for parameters and cell counts, focusing particularly on higher-dimensional problems. As a by-product, we note potential uses of the “global move” approach for inference about numbers of tables consistent with a prescribed subset of marginal counts. Illustration and comparison of MCMC approaches is given, and we conclude with discussion of areas for further developments and current open issues.     

Classical multilevel and Bayesian approaches to population size estimation using multiple lists

One of the major objections to the standard multiple-recapture approach to population estimation is the assumption of homogeneity of individual 'capture' probabilities. Modelling individual capture heterogeneity is complicated by the fact that it shows up as a restricted form of interaction among lists in the contingency table cross-classifying list memberships for all individuals. Traditional log-linear modelling approaches to capture–recapture problems are well suited to modelling interactions among lists but ignore the special dependence structure that individual heterogeneity induces. A random-effects approach, based on the Rasch model from educational testing and introduced in this context by Darroch and co-workers and Agresti, provides one way to introduce the dependence resulting from heterogeneity into the log-linear model; however, previous efforts to combine the Rasch-like heterogeneity terms additively with the usual log-linear interaction terms suggest that a more flexible approach is required. In this paper we consider both classical multilevel approaches and fully Bayesian hierarchical approaches to modelling individual heterogeneity and list interactions. Our framework encompasses both the traditional log-linear approach and various elements from the full Rasch model. We compare these approaches on two examples, the first arising from an epidemiological study of a population of diabetics in Italy, and the second a study intended to assess the 'size' of the World Wide Web. We also explore extensions allowing for interactions between the Rasch and log-linear portions of the models in both the classical and the Bayesian contexts.     

A Bayesian Approach for Zero-Inflated Count Regression Models by Using the Reversible Jump Markov Chain Monte Carlo Method and an Application

In this study, estimation of the parameters of the zero-inflated count regression models and computations of posterior model probabilities of the log-linear models defined for each zero-inflated count regression models are investigated from the Bayesian point of view. In addition, determinations of the most suitable log-linear and regression models are investigated. It is known that zero-inflated count regression models cover zero-inflated Poisson, zero-inflated negative binomial, and zero-inflated generalized Poisson regression models. The classical approach has some problematic points but the Bayesian approach does not have similar flaws. This work points out the reasons for using the Bayesian approach. It also lists advantages and disadvantages of the classical and Bayesian approaches. As an application, a zoological data set, including structural and sampling zeros, is used in the presence of extra zeros. In this work, it is observed that fitting a zero-inflated negative binomial regression model creates no problems at all, even though it is known that fitting a zero-inflated negative binomial regression model is the most problematic procedure in the classical approach. Additionally, it is found that the best fitting model is the log-linear model under the negative binomial regression model, which does not include three-way interactions of factors.     

Bivariate log Birnbaum–Saunders distribution

Univariate Birnbaum–Saunders distribution has received a considerable amount of attention in recent years. Rieck and Nedelman (A log-linear model for the Birnbaum–Saunders distribution. Technometrics, 1991;33:51–60) introduced a log Birnbaum–Saunders distribution. The main aim of this paper is to introduce bivariate log Birnbaum–Saunders distribution. The proposed model is symmetric and it has five parameters. It can be obtained using Gaussian copula. Different properties can be obtained using copula structure. It is observed that the maximum likelihood estimators (MLEs) cannot be obtained explicitly. Two-dimensional profile likelihood approach may be adopted to compute the MLEs. We propose some alternative estimators also, which can be obtained quite conveniently. The analysis of one data set is performed for illustrative purposes. Finally, it is observed that this model can be used as a bivariate log-linear model, and its multivariate generalization is also quite straight forward.     

A Bayesian Approach to the Estimation of Expected Cell Counts by Using Log-Linear Models

ABSTRACT

In this article, Bayesian estimation of the expected cell counts for log-linear models is considered. The prior specified for log-linear parameters is used to determine a prior for expected cell counts, by means of the family and parameters of prior distributions. This approach is more cost-effective than working directly with cell counts because converting prior information into a prior distribution on the log-linear parameters is easier than that of on the expected cell counts. While proceeding from the prior on log-linear parameters to the prior of the expected cell counts, we faced with a singularity problem of variance matrix of the prior distribution, and added a new precision parameter to solve the problem. A numerical example is also given to illustrate the usage of the new parameter.     

Constant Stress Accelerated Life Test on a Multiple-Component Series System under Weibull Lifetime Distributions

We will discuss the reliability analysis of the constant stress accelerated life test on a series system connected with multiple components under independent Weibull lifetime distributions whose scale parameters are log-linear in the level of the stress variable. The system lifetimes are collected under Type I censoring but the components that cause the systems to fail may or may not be observed. The data are so called masked for the latter case. Maximum likelihood approach and the Bayesian method are considered when the data are masked. Statistical inference on the estimation of the underlying model parameters as well as the mean time to failure and the reliability function will be addressed. Simulation study for a three-component case shows that Bayesian analysis outperforms the maximum likelihood approach especially when the data are highly masked.     

Collapsibility of contingency tables based on conditional models

Strict collapsibility and model collapsibility are two important concepts associated with the dimension reduction of a multidimensional contingency table, without losing the relevant information. In this paper, we obtain some necessary and sufficient conditions for the strict collapsibility of the full model, with respect to an interaction factor or a set of interaction factors, based on the interaction parameters of the conditional/layer log-linear models. For hierarchical log-linear models, we present also necessary and sufficient conditions for the full model to be model collapsible, based on the conditional interaction parameters. We discuss both the cases where one variable or a set of variables is conditioned. The connections between the strict collapsibility and the model collapsibility are also pointed out. Our results are illustrated through suitable examples, including a real life application.     

Maximum likelihood estimation for the negative multinomial log-linear model

A log-linear model is defined for multiway contingency tables with negative multinomial frequency counts. The maximum likelihood estimator of the model parameters and the estimator covariance matrix is given. The likelihood ratio test for the general log-linear hypothesis also is presented.     

Measuring and Estimating Treatment Effect on Count Outcome in Randomized Trial and Observational Studies

When estimating treatment effect on count outcome of given population, one uses different models in different studies, resulting in non-comparable measures of treatment effect. Here we show that the marginal rate differences in these studies are comparable measures of treatment effect. We estimate the marginal rate differences by log-linear models and show that their finite-sample maximum-likelihood estimates are unbiased and highly robust with respect to effects of dispersing covariates on outcome. We get approximate finite-sample distributions of these estimates by using the asymptotic normal distribution of estimates of the log-linear model parameters. This method can be easily applied to practice.     

Log-linear modelling of data from matched case-control studies

A log-linear modelling approach is proposed for dealing with polytomous, unordered exposure variables in case-control epidemiological studies with matched pairs. Hypotheses concerning epidemiological parameters are shown to be expressable in terms of log-linear models for the expected frequencies of the case-by-control square concordance table representation of the matched data; relevant maximum likelihood estimates and goodness-of-fit statistics are presented. Possible extensions to account for ordered categorical risk factors and multiple controls are illustrated, and comparisons with previous work are discussed. Finally, the possibility of implementing the proposed method with GLIM is illustrated within the context of a data set already analyzed by other authors.     

On constant stress accelerated life tests terminated by Type II censoring at one of the stress levels

This paper considers constant stress accelerated life tests terminated by a Type II censoring regime at one of the stress levels. We consider a model based on Weibull distributions with constant shape and a log-linear link between scale and the stress factor. We obtain expectations associated with the likelihood function, and use these to obtain asymptotically valid variances and correlations for maximum likelihood estimates of model parameters. We illustrate their calculation, and assess agreement with observed counterparts for finite samples in simulation experiments. We then use moments to compare the information obtained from variants of the design, and show that, with an appropriate allocation of items to stress levels, the design yields better estimates of model parameters and related quantities than a single stress experiment.     

Bayesian selection of log-linear models

A general methodology is presented for finding suitable Poisson log-linear models with applications to multiway contingency tables. Mixtures of multivariate normal distributions are used to model prior opinion when a subset of the regression vector is believed to be nonzero. This prior distribution is studied for two- and three-way contingency tables, in which the regression coefficients are interpretable in terms of odds ratios in the table. Efficient and accurate schemes are proposed for calculating the posterior model probabilities. The methods are illustrated for a large number of two-way simulated tables and for two three-way tables. These methods appear to be useful in selecting the best log-linear model and in estimating parameters of interest that reflect uncertainty in the true model.     

On a multivariate log-gamma distribution and the use of the distribution in the Bayesian analysis

In this article, we propose a new generalized multivariate log-gamma distribution. We consider the usage of the proposed multivariate distribution as the prior distribution in the Bayesian analysis. The generalized multivariate log-gamma distribution allows for the inclusion of prior knowledge on correlations between model parameters when likelihood is not in the form of a normal distribution. Use of the proposed distribution in the Bayesian analysis of log-linear models is also discussed.     

Relationships Between Full and Layer Models with Applications to Level Merging

Analysis of a large dimensional contingency table is quite involved. Models corresponding to layers of a contingency table are easier to analyze than the full model. Relationships between the interaction parameters of the full log-linear model and that of its corresponding layer models are obtained. These relationships are not only useful to reduce the analysis but also useful to interpret various hierarchical models. We obtain these relationships for layers of one variable, and extend the results for the case when layers of more than one variable are considered. We also establish, under conditional independence, relationships between the interaction parameters of the full model and that of the corresponding marginal models. We discuss the concept of merging of factor levels based on these interaction parameters. Finally, we use the relationships between layer models and full model to obtain conditions for level merging based on layer interaction parameters. Several examples are discussed to illustrate the results.     

On-Line Learning for the Infinite Hidden Markov Model

We develop a sequential Monte Carlo algorithm for the infinite hidden Markov model (iHMM) that allows us to perform on-line inferences on both system states and structural (static) parameters. The algorithm described here provides a natural alternative to Markov chain Monte Carlo samplers previously developed for the iHMM, and is particularly helpful in applications where data is collected sequentially and model parameters need to be continuously updated. We illustrate our approach in the context of both a simulation study and a financial application.     

Optimal step-stress test under type I progressive group-censoring with random removals

Some traditional life tests result in no or very few failures by the end of test. In such cases, one approach is to do life testing at higher-than-usual stress conditions in order to obtain failures quickly. This paper discusses a k-level step-stress accelerated life test under type I progressive group-censoring with random removals. An exponential failure time distribution with mean life that is a log-linear function of stress and a cumulative exposure model are considered. We derive the maximum likelihood estimators of the model parameters and establish the asymptotic properties of the estimators. We investigate four selection criteria which enable us to obtain the optimum test plans. One is to minimize the asymptotic variance of the maximum likelihood estimator of the logarithm of the mean lifetime at use-condition, and the other three criteria are to maximize the determinant, trace and the smallest eigenvalue of Fisher's information matrix. Some numerical studies are discussed to illustrate the proposed criteria.     

Maximum Likelihood Inference for Log-linear Models Subject to Constraints of Double Monotone Dependence

To model an hypothesis of double monotone dependence between two ordinal categorical variables A and B usually a set of symmetric odds ratios defined on the joint probability function is subject to linear inequality constraints. Conversely in this paper two sets of asymmetric odds ratios defined, respectively, on the conditional distributions of A given B and on the conditional distributions of B given A are subject to linear inequality constraints. If the joint probabilities are parameterized by a saturated log-linear model, these constraints are nonlinear inequality constraints on the log-linear parameters. The problem here considered is a non-standard one both for the presence of nonlinear inequality constraints and for the fact that the number of these constraints is greater than the number of the parameters of the saturated log-linear model.This work has been supported by the COFIN 2002 project, references 2002133957_002, 2002133957_004. Preliminary findings have been presented at SIS (Società Italiana di Statistica) Annual Meeting, Bari, 2004.     

Free Knot Splines with RJMCMC in Survival Data Analysis

The B-spline representation is a common tool to improve the fitting of smooth nonlinear functions, it offers a fitting as a piecewise polynomial. The regions that define the pieces are separated by a sequence of knots. The main difficulty in this type of modeling is the choice of the number and the locations of these knots. The Reversible Jump Markov Chain Monte Carlo (RJMCMC) algorithm provides a solution to simultaneously select these two parameters by considering the knots as free parameters. This algorithm belongs to the MCMC techniques that allow simulations from target distributions on spaces of varying dimension. The aim of the present investigation is to use this algorithm in the framework of the analysis of survival time, for the Cox model in particular. In fact, the relation between the hazard ratio function and the covariates being assumed to be log-linear, this assumption is too restrictive. Thus, we propose to use the RJMCMC algorithm to model the log hazard ratio function by a B-spline representation with an unknown number of knots at unknown locations. This method is illustrated with two real data sets: the Stanford heart transplant data and lung cancer survival data. Another application of the RJMCMC is selecting the significant covariates, and a simulation study is performed.     

Stochastic search variable selection for log-linear models

We develop a Markov chain Monte Carlo algorithm, based on ‘stochastic search variable selection’ (George and McCuUoch, 1993), for identifying promising log-linear models. The method may be used in the analysis of multi-way contingency tables where the set of plausible models is very large.