Partitioning Pearson's Chi-Squared Statistic for a Completely Ordered Three-Way Contingency Table

The paper presents a partition of the Pearson chi-squared statistic for triply ordered three-way contingency tables. The partition invokes orthogonal polynomials and identifies three-way association terms as well as each combination of two-way associations. This partition provides information about the structure of each variable by identifying important bivariate and trivariate associations in terms of location (linear), dispersion (quadratic) and higher order components. The significance of each term in the partition, and each association within each term can also be determined.
The paper compares the chi-squared partition with the log-linear models of Agresti (1994) for multi-way contingency tables with ordinal categories, by generalizing the model proposed by Haberman (1974).     

Bayesian methods for contingency tables using Gibbs sampling

Cell counts in contingency tables can be smoothed using loglinear models. Recently, sampling-based methods such as Markov chain Monte Carlo (MCMC) have been introduced, making it possible to sample from posterior distributions. The novelty of the approach presented here is that all conditional distributions can be specified directly, so that straight-forward Gibbs sampling is possible. Thus, the model is constructed in a way that makes burn-in and checking convergence a relatively minor issue. The emphasis of this paper is on smoothing cell counts in contingency tables, and not so much on estimation of regression parameters. Therefore, the prior distribution consists of two stages. We rely on a normal nonconjugate prior at the first stage, and a vague prior for hyperparameters at the second stage. The smoothed counts tend to compromise between the observed data and a log-linear model. The methods are demonstrated with a sparse data table taken from a multi-center clinical trial. The research for the first author was supported by Brain Pool program of the Korean Federation of Science and Technology Societies. The research for the second author was partially supported by KOSEF through Statistical Research Center for Complex Systems at Seoul National University.     

Independence in multi-way contingency tables: S.N. Roy's breakthroughs and later developments

In the mid-1950s S.N. Roy and his students contributed two landmark articles to the contingency table literature [Roy, S.N., Kastenbaum, M.A., 1956. On the hypothesis of no “interaction” in a multiway contingency table. Ann. Math. Statist. 27, 749–757; Roy, S.N., Mitra, S.K., 1956. An introduction to some nonparametric generalizations of analysis of variance and multivariate analysis. Biometrika 43, 361–376]. The first article generalized concepts of interaction from 2×2×2$2\times 2\times 2$ contingency tables to three-way tables of arbitrary size and to larger tables. In the second article, which is the source of our primary focus, various notions of independence were clarified for three-way contingency tables, Roy's union–intersection test was applied to construct chi-squared tests of hypotheses about the structure of such tables, and the chi-squared statistics were shown not to depend on the distinction between response and explanatory variables. This work pre-dates by many years later developments that expressed such results in the context of loglinear models. It pre-dates by a quarter century the development of graphical models. We summarize the main results in these key articles and discuss the connection between them and the later developments of loglinear modeling and of graphical modeling. We also mention ways in which these later developments have themselves been further generalized.     

Data augmentation in multi-way contingency tables with fixed marginal totals

We describe and illustrate approaches to data augmentation in multi-way contingency tables for which partial information, in the form of subsets of marginal totals, is available. In such problems, interest lies in questions of inference about the parameters of models underlying the table together with imputation for the individual cell entries. We discuss questions of structure related to the implications for inference on cell counts arising from assumptions about log-linear model forms, and a class of simple and useful prior distributions on the parameters of log-linear models. We then discuss “local move” and “global move” Metropolis–Hastings simulation methods for exploring the posterior distributions for parameters and cell counts, focusing particularly on higher-dimensional problems. As a by-product, we note potential uses of the “global move” approach for inference about numbers of tables consistent with a prescribed subset of marginal counts. Illustration and comparison of MCMC approaches is given, and we conclude with discussion of areas for further developments and current open issues.     

Maximum likelihood estimation for the negative multinomial log-linear model

A log-linear model is defined for multiway contingency tables with negative multinomial frequency counts. The maximum likelihood estimator of the model parameters and the estimator covariance matrix is given. The likelihood ratio test for the general log-linear hypothesis also is presented.     

Log-linear models for mutations in the HIV genome

We discuss a general application of categorical data analysis to mutations along the HIV genome. We consider a multidimensional table for several positions at the same time. Due to the complexity of the multidimensional table, we may collapse it by pooling some categories. However, the association between the remaining variables may not be the same as before collapsing. We discuss the collapsibility of tables and the change in the meaning of parameters after collapsing categories. We also address this problem with a log-linear model. We present a parameterization with the consensus output as the reference cell as is appropriate to explain genomic mutations in HIV. We also consider five null hypotheses and some classical methods to address them. We illustrate methods for six positions along the HIV genome, through consideration of all triples of positions.     

Independence in Contingency Tables Using Simplicial Geometry

Frequently, contingency tables are generated in a multinomial sampling. Multinomial probabilities are then organized in a table assigning probabilities to each cell. A probability table can be viewed as an element in the simplex. The Aitchison geometry of the simplex identifies independent probability tables as a linear subspace. An important consequence is that, given a probability table, the nearest independent table is obtained by orthogonal projection onto the independent subspace. The nearest independent table is identified as that obtained by the product of geometric marginals, which do not coincide with the standard marginals, except in the independent case. The original probability table is decomposed into orthogonal tables, the independent and the interaction tables. The underlying model is log-linear, and a procedure to test independence of a contingency table, based on a multinomial simulation, is developed. Its performance is studied on an illustrative example.     

Some simple log-linear models for the analysis of ordinal response data

A family of log-linear models are proposed to describe contingency tables in which one variable can be considered as the response to the remaining. The proposed models take into account the ordering nature of the response categories and have structure similar to that employed in polynomial regression. Stochastic ordering of the response distributions under the proposed models is discussed and the model-reduction techniques are developed. The proposed models are applied to two data sets previously analysed in the literature.     

Decomposing three dimensional contingency tables

A simple box-like diagram makes the task of teaching or analyzing three dimensional contingency tables much easier. Little is to be gained using the log-linear model and iterative procedures, as all but one hypothesis can be tested and necessary statistics calculated with a hand-held electronic calculator and without iterations.     

Catanova for multidimensional contingency tables: Nominal-scale response

This paper extends an analysis of variance for categorical data (CATANOVA) procedure to multidimensional contingency tables involving several factors and a response variable measured on a nominal scale. Using an appropriate measure of total variation for multinomial data, partial and multiple association measures are developed as R² quantities which parallel the analogous statistics in multiple linear regression for quantitative data. In addition, test statistics are derived in terms of these R² criteria. Finally, this CATANOVA approach is illustrated within the context of 2 three-way contingency table from a multicenter clinicaltrial.     

Semiparametric smoothing of sparse contingency tables

In the paper simple resampling technique based on semiparametric smoothing is introduced. Although the method is very flexible and in principle can be applied to any sparse data and ill-posed statistical problem, its efficient or even reasonable implementation requires special investigation. In the paper a problem of fitting local dependence structure of finite-state random sequences is addressed. This problem is relevant, for example, in genetics, bioinformatics, computer linguistics, etc., and usually leads to analysis of sparse contingency tables of dependent categorical data. Thus, the classical assumptions of log-linear model, a standard technique for analysis of contingency tables, do not hold. A framework convenient for implementation of semiparametric smoothing and resampling is proposed. It is based on a special representation form of data under consideration and generalized logit model. A computer experiment is carried out to gain better insight on practical performance of the procedure.     

Evaluation of missing data mechanisms in two and three dimensional incomplete tables

The analysis of incomplete contingency tables is a practical and an interesting problem. In this paper, we provide characterizations for the various missing mechanisms of a variable in terms of response and non-response odds for two and three dimensional incomplete tables. Log-linear parametrization and some distinctive properties of the missing data models for the above tables are discussed. All possible cases in which data on one, two or all variables may be missing are considered. We study the missingness of each variable in a model, which is more insightful for analyzing cross-classified data than the missingness of the outcome vector. For sensitivity analysis of the incomplete tables, we propose easily verifiable procedures to evaluate the missing at random (MAR), missing completely at random (MCAR) and not missing at random (NMAR) assumptions of the missing data models. These methods depend only on joint and marginal odds computed from fully and partially observed counts in the tables, respectively. Finally, some real-life datasets are analyzed to illustrate our results, which are confirmed based on simulation studies.     

Log-linear model analysis of the semi-symmetric intraclass contingency table

This paper addresses the problem of analyzing a three-way contingency table that is upper-triangular, and a priori symmetric within layers. The log-linear model is modified to handle this kind of table, and maximum likelihood estimation is carried out for the modified log-linear model. This leads to an expression of the maximum likelihood estimates exclusively in terms of the observed cell counts. It is skin this analysis is equivalent to an application of the gone log-linear model to an artificially complete table, obtain. by splitting the off-diagonal cells in half within layers. This analysis is used in analyzing the results of a study done to determine the effect of the sex-linked dwarfing gene in male chickens on resistance to E. coli infection; the conclusion differs from that of a previous analysis of the same data (see Norwood and Hinkelmann 1978). It is found, in fact, that the structure of association among the two allele variables and the disease variable is somewhat more complex than previously proposed. A second example is taken from Ishii (1960). Finally, collapsibility conditions for the modified log-linear model, as well as various other sampling plans and limitations to the testing procedure, are discussed.     

Bayesian inference for Poisson and multinomial log-linear models

Categorical data frequently arise in applications in the Social Sciences. In such applications, the class of log-linear models, based on either a Poisson or (product) multinomial response distribution, is a flexible model class for inference and prediction. In this paper we consider the Bayesian analysis of both Poisson and multinomial log-linear models. It is often convenient to model multinomial or product multinomial data as observations of independent Poisson variables. For multinomial data, Lindley (1964) [20] showed that this approach leads to valid Bayesian posterior inferences when the prior density for the Poisson cell means factorises in a particular way. We develop this result to provide a general framework for the analysis of multinomial or product multinomial data using a Poisson log-linear model. Valid finite population inferences are also available, which can be particularly important in modelling social data. We then focus particular attention on multivariate normal prior distributions for the log-linear model parameters. Here, an improper prior distribution for certain Poisson model parameters is required for valid multinomial analysis, and we derive conditions under which the resulting posterior distribution is proper. We also consider the construction of prior distributions across models, and for model parameters, when uncertainty exists about the appropriate form of the model. We present classes of Poisson and multinomial models, invariant under certain natural groups of permutations of the cells. We demonstrate that, if prior belief concerning the model parameters is also invariant, as is the case in a ‘reference’ analysis, then the choice of prior distribution is considerably restricted. The analysis of multivariate categorical data in the form of a contingency table is considered in detail. We illustrate the methods with two examples.     

Partitioning chi-squape in contingency tables: A teaching approach

A representation of sums and differences of the form 2n log n, the lnn function, is introduced to express likelihood-ratio chi-square test statistics in contingency table analysis. This is a concise explicit form to display when partitioning chi-square statistics in accordance with hierarchical models. The lnn representation gives students insights into the construction of test statistics, and assists in relating identical forms under differing model sets. Hierarchies are presented for independence and equi-probability in two-way tables, for symmetry in correlated square tables, for independence-and-homogeneity of two-way responses across levels of a factor, and for mutual independence in three-way tables, along with relevant partitions of chi-square.     

Stochastic search variable selection for log-linear models

We develop a Markov chain Monte Carlo algorithm, based on ‘stochastic search variable selection’ (George and McCuUoch, 1993), for identifying promising log-linear models. The method may be used in the analysis of multi-way contingency tables where the set of plausible models is very large.     

A Bayesian Approach for Zero-Inflated Count Regression Models by Using the Reversible Jump Markov Chain Monte Carlo Method and an Application

In this study, estimation of the parameters of the zero-inflated count regression models and computations of posterior model probabilities of the log-linear models defined for each zero-inflated count regression models are investigated from the Bayesian point of view. In addition, determinations of the most suitable log-linear and regression models are investigated. It is known that zero-inflated count regression models cover zero-inflated Poisson, zero-inflated negative binomial, and zero-inflated generalized Poisson regression models. The classical approach has some problematic points but the Bayesian approach does not have similar flaws. This work points out the reasons for using the Bayesian approach. It also lists advantages and disadvantages of the classical and Bayesian approaches. As an application, a zoological data set, including structural and sampling zeros, is used in the presence of extra zeros. In this work, it is observed that fitting a zero-inflated negative binomial regression model creates no problems at all, even though it is known that fitting a zero-inflated negative binomial regression model is the most problematic procedure in the classical approach. Additionally, it is found that the best fitting model is the log-linear model under the negative binomial regression model, which does not include three-way interactions of factors.     

Three centuries of categorical data analysis: Log-linear models and maximum likelihood estimation

The common view of the history of contingency tables is that it begins in 1900 with the work of Pearson and Yule, but in fact it extends back at least into the 19th century. Moreover, it remains an active area of research today. In this paper we give an overview of this history focussing on the development of log-linear models and their estimation via the method of maximum likelihood. Roy played a crucial role in this development with two papers co-authored with his students, Mitra and Marvin Kastenbaum, at roughly the mid-point temporally in this development. Then we describe a problem that eluded Roy and his students, that of the implications of sampling zeros for the existence of maximum likelihood estimates for log-linear models. Understanding the problem of non-existence is crucial to the analysis of large sparse contingency tables. We introduce some relevant results from the application of algebraic geometry to the study of this statistical problem.     

Mutual information and redundancy for categorical data

Most methods for describing the relationship among random variables require specific probability distributions and some assumptions concerning random variables. Mutual information, based on entropy to measure the dependency among random variables, does not need any specific distribution and assumptions. Redundancy, which is an analogous version of mutual information, is also proposed as a method. In this paper, the concepts of redundancy and mutual information are explored as applied to multi-dimensional categorical data. We found that mutual information and redundancy for categorical data can be expressed as a function of the generalized likelihood ratio statistic under several kinds of independent log-linear models. As a consequence, mutual information and redundancy can also be used to analyze contingency tables stochastically. Whereas the generalized likelihood ratio statistic to test the goodness-of-fit of the log-linear models is sensitive to the sample size, the redundancy for categorical data does not depend on sample size but depends on its cell probabilities.     

Asymptotic cumulants of the minimum phi-divergence estimator for categorical data under possible model misspecification

Abstract

The asymptotic cumulants of the minimum phi-divergence estimators of the parameters in a model for categorical data are obtained up to the fourth order with the higher-order asymptotic variance under possible model misspecification. The corresponding asymptotic cumulants up to the third order for the studentized minimum phi-divergence estimator are also derived. These asymptotic cumulants, when a model is misspecified, depend on the form of the phi-divergence. Numerical illustrations with simulations are given for typical cases of the phi-divergence, where the maximum likelihood estimator does not necessarily give best results. Real data examples are shown using log-linear models for contingency tables.