Simulated classical tests in multinomial probit models

This paper compares the application of different versions of the simulated counterparts of the Wald test, the score test, and the likelihood ratio test in one- and multiperiod multinomial probit models. Monte Carlo experiments show that the use of the simple form of the simulated likelihood ratio test delivers relatively robust results regarding the testing of several multinomial probit model specifications. In contrast, the inclusion of the Hessian matrix of the simulated loglikelihood function into the simulated score test and (in the multiperiod multinomial probit model) particularly the inclusion of the quasi-maximum likelihood theory into the simulated likelihood ratio test leads to substantial computational problems. The combined application of the quasi-maximum likelihood theory with the simulated Wald test or the simulated score test is not systematically superior to the application of the other versions of these two simulated classical tests either. Neither an increase in the number of observations nor in the number of random draws in the incorporated Geweke-Hajivassiliou-Keane simulator systematically lead to more precise conformities between the frequencies of type I errors and the basic significance levels. An increase in the number of observations only decreases the frequencies of type II errors, particularly regarding the simulated classical testing of multiperiod multinomial probit model specifications.     

A self-consistency approach to multinomial logit model with random effects

The computation in the multinomial logit mixed effects model is costly especially when the response variable has a large number of categories, since it involves high-dimensional integration and maximization. Tsodikov and Chefo (2008) developed a stable MLE approach to problems with independent observations, based on generalized self-consistency and quasi-EM algorithm developed in Tsodikov (2003). In this paper, we apply the idea to clustered multinomial response to simplify the maximization step. The method transforms the complex multinomial likelihood to Poisson-type likelihood and hence allows for the estimates to be obtained iteratively solving a set of independent low-dimensional problems. The methodology is applied to real data and studied by simulations. While maximization is simplified, numerical integration remains the dominant challenge to computational efficiency.     

Posterior Distributions for the Gini Coefficient Using Grouped Data

When available data comprise a number of sampled households in each of a number of income classes, the likelihood function is obtained from a multinomial distribution with the income class population proportions as the unknown parameters. Two methods for going from this likelihood function to a posterior distribution on the Gini coefficient are investigated. In the first method, two alternative assumptions about the underlying income distribution are considered, namely a lognormal distribution and the Singh–Maddala (1976) income distribution. In these cases the likelihood function is reparameterized and the Gini coefficient is a nonlinear function of the income distribution parameters. The Metropolis algorithm is used to find the corresponding posterior distributions of the Gini coefficient from a sample of Bangkok households. The second method does not require an assumption about the nature of the income distribution, but uses (a) triangular prior distributions, and (b) beta prior distributions, on the location of mean income within each income class. By sampling from these distributions, and the Dirichlet posterior distribution of the income class proportions, alternative posterior distributions of the Gini coefficient are calculated.     

Labelled Graphical Models

A class of log‐linear models, referred to as labelled graphical models (LGMs), is introduced for multinomial distributions. These models generalize graphical models (GMs) by employing partial conditional independence restrictions which are valid only in subsets of an outcome space. Theoretical results concerning model identifiability, decomposability and estimation are derived. A decision theoretical framework and a search algorithm for the identification of plausible models are described. Real data sets are used to illustrate that LGMs may provide a simpler interpretation of a dependence structure than GMs.     

Clustering Discrete Data Through the Multinomial Mixture Model

In this work, the multinomial mixture model is studied, through a maximum likelihood approach. The convergence of the maximum likelihood estimator to a set with characteristics of interest is shown. A method to select the number of mixture components is developed based on the form of the maximum likelihood estimator. A simulation study is then carried out to verify its behavior. Finally, two applications on real data of multinomial mixtures are presented.     

Inadmissibility results of the mle for the multinomial problem when the parameter space is restricted or truncated

In binomial or multinomial problems when the parameter space is restricted or truncated to a subset of the natural parameter space, the maximum likelihood estimator (MLE) may be inadmissible under squared error loss. A quite general condition for the inadmissibility of MLEs in such cases can be established using the stepwise Bayes technique and the complete class theorem of Brown.     

Bayesian inference for Poisson and multinomial log-linear models

Categorical data frequently arise in applications in the Social Sciences. In such applications, the class of log-linear models, based on either a Poisson or (product) multinomial response distribution, is a flexible model class for inference and prediction. In this paper we consider the Bayesian analysis of both Poisson and multinomial log-linear models. It is often convenient to model multinomial or product multinomial data as observations of independent Poisson variables. For multinomial data, Lindley (1964) [20] showed that this approach leads to valid Bayesian posterior inferences when the prior density for the Poisson cell means factorises in a particular way. We develop this result to provide a general framework for the analysis of multinomial or product multinomial data using a Poisson log-linear model. Valid finite population inferences are also available, which can be particularly important in modelling social data. We then focus particular attention on multivariate normal prior distributions for the log-linear model parameters. Here, an improper prior distribution for certain Poisson model parameters is required for valid multinomial analysis, and we derive conditions under which the resulting posterior distribution is proper. We also consider the construction of prior distributions across models, and for model parameters, when uncertainty exists about the appropriate form of the model. We present classes of Poisson and multinomial models, invariant under certain natural groups of permutations of the cells. We demonstrate that, if prior belief concerning the model parameters is also invariant, as is the case in a ‘reference’ analysis, then the choice of prior distribution is considerably restricted. The analysis of multivariate categorical data in the form of a contingency table is considered in detail. We illustrate the methods with two examples.     

Overview of multinomial models for ordinal data

The purpose of this paper is to relate a number of multinomial models currently in use for ordinal response data in a unified manner. By studying generalized logit models, proportional generalized odds ratio models and proportional generalized hazard models under different parameterizations, we conclude that there are only four different models and they can be specified genericaUy in a uniform way. These four models all possess the same stochastic ordering property and we compare them graphically in a simple case. Data from the NHLBI TYPE II study (Brensike et al (1984)) is used to illustrate these models. We show that the BMDP programs LE and PR can be employed in computing maximum likelihood estimators for these four models.     

An L1-type estimator of multivariate location and shape

The main goal of the paper is to specify a suitable multivariate multilevel model for polytomous responses with a non-ignorable missing data mechanism in order to determine the factors which influence the way of acquisition of the skills of the graduates and to evaluate the degree programmes on the basis of the adequacy of the skills they give to their graduates. The application is based on data gathered by a telephone survey conducted, about two years after the degree, on the graduates of year 2000 of the University of Florence. A multilevel multinomial logit model for the response of interest is fitted simultaneously with a multilevel logit model for the selection mechanism by means of maximum likelihood with adaptive Gaussian quadrature. In the application the multilevel structure has a crucial role, while selection bias results negligible. The analysis of the empirical Bayes residuals allows to detect some extreme degree programmes to be further inspected.     

A comparison of artificial neural network and multinomial logit models in predicting mergers

A merger proposal discloses a bidder firm's desire to purchase the control rights in a target firm. Predicting who will propose (bidder candidacy) and who will receive (target candidacy) merger bids is important to investigate why firms merge and to measure the price impact of mergers. This study investigates the performance of artificial neural networks and multinomial logit models in predicting bidder and target candidacy. We use a comprehensive data set that covers the years 1979–2004 and includes all deals with publicly listed bidders and targets. We find that both models perform similarly while predicting target and non-merger firms. The multinomial logit model performs slightly better in predicting bidder firms.     

Modelling mass-size particle data by finite mixtures

A minimum distance procedure, analogous to maximum likelihood for multinomial data, is employed to fit mixture models to mass-size relative frequencies recorded for some clay soils of southeastern Australia. Log hyperbolic component distributions are considered initially and it is shown how they can be fitted satisfactorily at least to ungrouped data using a generalized EM algorithm. A computationally more convenient model with log skew Laplace components is subsequently shown to suffice. It is demonstrated how it can be fitted to the data in their original grouped form. Consideration is given also to the provision of standard errors using the idea of a quasi-sample size.     

Fuzzy clustering algorithm for latent class model

The expectation maximization (EM) algorithm is a widely used parameter approach for estimating the parameters of multivariate multinomial mixtures in a latent class model. However, this approach has unsatisfactory computing efficiency. This study proposes a fuzzy clustering algorithm (FCA) based on both the maximum penalized likelihood (MPL) for the latent class model and the modified penalty fuzzy c-means (PFCM) for normal mixtures. Numerical examples confirm that the FCA-MPL algorithm is more efficient (that is, requires fewer iterations) and more computationally effective (measured by the approximate relative ratio of accurate classification) than the EM algorithm.     

Inference for comparing a multinomial distribution with a known standard

We propose a new type of stochastic ordering which imposes a monotone tendency in differences between one multinomial probability and a known standard one. An estimation procedure is proposed for the constrained maximum likelihood estimate, and then the asymptotic null distribution is derived for the likelihood ratio test statistic for testing equality of two multinomial distributions against the new stochastic ordering. An alternative test is also discussed based on Neyman modified minimum chi-square estimator. These tests are illustrated with a set of heart disease data.     

Exact and Monte Carlo calculations of integrated likelihoods for the latent class model

The latent class model or multivariate multinomial mixture is a powerful approach for clustering categorical data. It uses a conditional independence assumption given the latent class to which a statistical unit is belonging. In this paper, we exploit the fact that a fully Bayesian analysis with Jeffreys non-informative prior distributions does not involve technical difficulty to propose an exact expression of the integrated complete-data likelihood, which is known as being a meaningful model selection criterion in a clustering perspective. Similarly, a Monte Carlo approximation of the integrated observed-data likelihood can be obtained in two steps: an exact integration over the parameters is followed by an approximation of the sum over all possible partitions through an importance sampling strategy. Then, the exact and the approximate criteria experimentally compete, respectively, with their standard asymptotic BIC approximations for choosing the number of mixture components. Numerical experiments on simulated data and a biological example highlight that asymptotic criteria are usually dramatically more conservative than the non-asymptotic presented criteria, not only for moderate sample sizes as expected but also for quite large sample sizes. This research highlights that asymptotic standard criteria could often fail to select some interesting structures present in the data.     

Contingency tables with structural zeros

In this paper, the method of Hocking and Oxspring (1971) to estimate multinomial probabilities when full and partial data are available for some cells is extended to estimate the cell probabilities of a contingency table with structural zeros. The estimates are maximum likelihood, and the process is sequential. The gain in precision is due to the use of partial data and the bias of the estimates is also investigated.     

Estimation of weighted multinomial probabilities under log-convex constraints

Many problems in Statistics involve maximizing a multinomial likelihood over a restricted region. In this paper, we consider instead maximizing a weighted multinomial likelihood. We show that a dual problem always exits which is frequently more tractable and that a solution to the dual problem leads directly to a solution of the primal problem. Moreover, the form of the dual problem suggests an iterative algorithm for solving the MLE problem when the constraint region can be written as a finite intersection of cones. We show that this iterative algorithm is guaranteed to converge to the true solution and show that when the cones are isotonic, this algorithm is a version of Dykstra's algorithm (Dykstra, J. Amer. Statist. Assoc. 78 (1983) 837–842) for the special case of least squares projection onto the intersection of isotonic cones. We give several meaningful examples to illustrate our results. In particular, we obtain the nonparametric maximum likelihood estimator of a monotone density function in the presence of selection bias.     

Maximum likelihood estimation for incomplete multinomial data via the weaver algorithm

In a multinomial model, the sample space is partitioned into a disjoint union of cells. The partition is usually immutable during sampling of the cell counts. In this paper, we extend the multinomial model to the incomplete multinomial model by relaxing the constant partition assumption to allow the cells to be variable and the counts collected from non-disjoint cells to be modeled in an integrated manner for inference on the common underlying probability. The incomplete multinomial likelihood is parameterized by the complete-cell probabilities from the most refined partition. Its sufficient statistics include the variable-cell formation observed as an indicator matrix and all cell counts. With externally imposed structures on the cell formation process, it reduces to special models including the Bradley–Terry model, the Plackett–Luce model, etc. Since the conventional method, which solves for the zeros of the score functions, is unfruitful, we develop a new approach to establishing a simpler set of estimating equations to obtain the maximum likelihood estimate (MLE), which seeks the simultaneous maximization of all multiplicative components of the likelihood by fitting each component into an inequality. As a consequence, our estimation amounts to solving a system of the equality attainment conditions to the inequalities. The resultant MLE equations are simple and immediately invite a fixed-point iteration algorithm for solution, which is referred to as the weaver algorithm. The weaver algorithm is short and amenable to parallel implementation. We also derive the asymptotic covariance of the MLE, verify main results with simulations, and compare the weaver algorithm with an MM/EM algorithm based on fitting a Plackett–Luce model to a benchmark data set.     

Properties of Bayes Factors Based on Test Statistics

Abstract.  This article examines the consistency, interpretation and application of Bayes factors constructed from standard test statistics. Primary conclusions are that Bayes factors based on multinomial and normal test statistics are consistent for suitable choices of the hyperparameters used to specify alternative hypotheses, and that such constructions can be extended to obtain consistent Bayes factors based on likelihood ratio statistics. A connection between Bayes factors based on likelihood ratio statistics and the Bayesian information criterion is exposed, as is a connection between Bayes factors based on F statistics and parametric Bayes factors based on normal-inverse gamma models. Similarly, Bayes factors based on chi-squared statistics for multinomial data are shown to provide accurate approximations to Bayes factors based on multinomial/Dirichlet models. An illustration of how the simple form of these Bayes factors can be exploited to generate easily interpretable summaries of the experimental 'weight of evidence' is provided.     

S–S method for stochastically ordered multinomial populations with missing data

Stochastic ordering is a useful concept in order restricted inferences. In this paper, we propose a new estimation technique for the parameters in two multinomial populations under stochastic orderings when missing data are present. In comparison with traditional maximum likelihood estimation method, our new method can guarantee the uniqueness of the maximum of the likelihood function. Furthermore, it does not depend on the choice of initial values for the parameters in contrast to the EM algorithm. Finally, we give the asymptotic distributions of the likelihood ratio statistics based on the new estimation method.