Flexible Covariance Structures for Categorical Dependent Variables Through Finite Mixtures of Generalized Extreme Value Models

A new class of finite mixture discrete choice models, denoted FinMix (fīn m?ks), is introduced. These arise from the combination of a finite number of core Generalized Extreme Value (GEV) models to achieve more flexible functional forms, particularly in terms of error covariance structures. Example members of the class include combinations of (1) Multinomial Logit (MNL) models with differing scales, (2) multinomial logit with nested MNL models, (3) tree extreme value models with differing preference trees, and so on. Compatibility of FinMix models with utility maximization is easily determined, which permits empirical investigation of the suitability of specific model forms for economic evaluation exercises.     

Inference for Mixtures

In this article, we consider the problem of estimating certain “parameters” in a mixture of probability measures. We show that a single sample is typically suitable for estimating the component measures, but not suitable for estimating the mixing measures, especially when consistency is required. To have consistent estimators of the mixing measure, several samples with increasing size are needed in general.     

Mixtures of Mixtures Based on Multiresolution Analysis Theory

In this article, the approaches for exploiting mixtures of mixtures are expanded by using the Multiresolution family of probability density functions (MR pdf). The flexibility and the properties of local analysis of the MR pdf facilitate the location of subpopulations into a given population. In order to do this, two algorithms are provided.

The MR model is more flexible in adapting to the different subpopulations than the traditional mixtures. In addition, the problems of identification of mixtures distributions and the label-switching do not appear in the MR pdf context.     

Distributions with Generalized Skewed Conditionals and Mixtures of Such Distributions

Mixtures of skewed distributions (univariate and bivariate) provide flexible models. An alternative modeling approach involves distributions with skewed conditional distributions and mixtures of such distributions. We consider the interrelationships between such models. Examples are provided to show that several skewed distributions already considered in the literature can be viewed as having been constructed via a combination of mixing and skewing.     

Model-Based Classification via Mixtures of Multivariate t-Factor Analyzers

A model-based classification technique is developed, based on mixtures of multivariate t-factor analyzers. Specifically, two related mixture models are developed and their classification efficacy studied. An AECM algorithm is used for parameter estimation, and convergence of these algorithms is determined using Aitken's acceleration. Two different techniques are proposed for model selection: the BIC and the ICL. Our classification technique is applied to data on red wine samples from Italy and to fatty acid measurements on Italian olive oils. These results are discussed and compared to more established classification techniques; under this comparison, our mixture models give excellent classification performance.     

Mixtures of Linear Regression with Measurement Errors

Existing research on mixtures of regression models are limited to directly observed predictors. The estimation of mixtures of regression for measurement error data imposes challenges for statisticians. For linear regression models with measurement error data, the naive ordinary least squares method, which directly substitutes the observed surrogates for the unobserved error-prone variables, yields an inconsistent estimate for the regression coefficients. The same inconsistency also happens to the naive mixtures of regression estimate, which is based on the traditional maximum likelihood estimator and simply ignores the measurement error. To solve this inconsistency, we propose to use the deconvolution method to estimate the mixture likelihood of the observed surrogates. Then our proposed estimate is found by maximizing the estimated mixture likelihood. In addition, a generalized EM algorithm is also developed to find the estimate. The simulation results demonstrate that the proposed estimation procedures work well and perform much better than the naive estimates.     

Information,ancillarity, and sufficiency in the presence of nuisance parameters

The Fisher information about parameters of interest (P-information) is invariant with respect to nuisance parameters, and induces an information inequality associated with likelihood factorization. This information inequality provides a natural basis for measuring information loss due to using only a sublikelihood function for inference. In contrast with the global reparametrization of some previous concepts in the literature, the concepts of P-ancillarity and P-sufficiency proposed in this article are characterized by the notion of no pointwise information loss with respect to the parameters of interest. A conditional version of P-sufficiency is also proposed. The asymptotic efficiency of likelihood inference under P-ancillarity or P-sufficiency is outlined.     

Mixtures of general location model with factor analyzer covariance structure for clustering mixed type data

Cluster analysis is one of the most widely used method in statistical analyses, in which homogeneous subgroups are identified in a heterogeneous population. Due to the existence of the continuous and discrete mixed data in many applications, so far, some ordinary clustering methods such as, hierarchical methods, k-means and model-based methods have been extended for analysis of mixed data. However, in the available model-based clustering methods, by increasing the number of continuous variables, the number of parameters increases and identifying as well as fitting an appropriate model may be difficult. In this paper, to reduce the number of the parameters, for the model-based clustering mixed data of continuous (normal) and nominal data, a set of parsimonious models is introduced. Models in this set are extended, using the general location model approach, for modeling distribution of mixed variables and applying factor analyzer structure for covariance matrices. The ECM algorithm is used for estimating the parameters of these models. In order to show the performance of the proposed models for clustering, results from some simulation studies and analyzing two real data sets are presented.     

An Approximate Bayesian Marginal Likelihood Approach for Estimating Finite Mixtures

Estimation of finite mixture models when the mixing distribution support is unknown is an important problem. This article gives a new approach based on a marginal likelihood for the unknown support. Motivated by a Bayesian Dirichlet prior model, a computationally efficient stochastic approximation version of the marginal likelihood is proposed and large-sample theory is presented. By restricting the support to a finite grid, a simulated annealing method is employed to maximize the marginal likelihood and estimate the support. Real and simulated data examples show that this novel stochastic approximation and simulated annealing procedure compares favorably with existing methods.     

Size and Power of Recent Tests for Homogeneity in Exponential Mixtures

The article investigates diagnostic procedures for finite mixture models. The problem is to decide whether given data stem from an exponential distribution or a finite mixture of such distributions. Recently, three new test approaches have been proposed, the modified likelihood ratio test (MLRT) by Chen et al. (2001 Chen , H. , Chen , J. , Kalbfleisch , J. D. ( 2001 ). A modified likelihood ratio test for homogeneity in finite mixture models . Journal of the Royal Statistical Society, B 63 : 19 – 29 .[Crossref] , [Google Scholar]), the ADDS test by Mosler and Seidel (2001 Mosler , K. , Seidel , W. ( 2001 ). Testing for homogeneity in an exponential mixture model . Australian and New Zealand Journal of Statistics 43 : 231 – 247 . [Google Scholar]), and the D-test by Charnigo and Sun (2004 Charnigo , R. , Sun , J. ( 2004 ). Testing homogeneity in a mixture distribution via the l ² distance between competing models . Journal of the American Statistical Society 99 : 488 – 498 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). The size and power of these tests are determined by Monte Carlo simulation and their relative merits are evaluated. We conclude that the ADDS test shows always not much less and under some alternatives, in particular lower contaminations, considerably more power than its competitors. Also, new tables for the ADDS test are provided.     

Adaptive Mixtures of Regressions: Improving Predictive Inference when Population has Changed

The present work investigates the estimation of regression mixtures when population has changed between the training and the prediction stages. Two approaches are proposed: a parametric approach modeling the relationship between dependent variables of both populations, and a Bayesian approach in which the priors on the prediction population depend on the mixture regression parameters of the training population. The relevance of both approaches is illustrated on simulations and on an environmental dataset.     

Book Reviews

Aggregate forecasts using McFadden's conditional logit model of discrete choice harbor the unrealistic implicit assumption of a random-utility distribution that is homogeneous across both alternatives and individuals. This article relaxes that assumption. A choice model is developed that describes the random-utility component as a probability-mixture model. Some numerical results illustrate that the derived model is not constrained by the independence-of-irrelevant-alternatives property. An experimental test of visual perceptions demonstrates the potential superiority of the model.     

Identifying Structural Models of Committee Decisions With Heterogeneous Tastes and Ideological Bias

In practice, members of a committee often make different recommendations despite a common goal and shared sources of information. We study the nonparametric identification and estimation of a structural model, where such discrepancies are rationalized by the members’ unobserved types, which consist of ideological bias while weighing different sources of information, and tastes for multiple objectives announced in the policy target. We consider models with and without strategic incentives for members to make recommendations that conform to the final committee decision. We show that pure-strategy Bayesian Nash equilibria exist in both cases, and that the variation in common information recorded in the data helps us to recover the distribution of private types from the members’ choices. Building on the identification result, we estimate a structural model of interest rate decisions by the Monetary Policy Committee (MPC) at the Bank of England. We find some evidence that the external committee members are less affected by strategic incentives for conformity in their recommendations than the internal members. We also find that the difference in ideological bias between external and internal members is statistically insignificant. Supplementary materials for this article are available online.     

Nonparametric estimation of a change in defect intensity

Summary.  In an important class of problems involving mixture distributions, interest focuses on the mixture proportions, considering other possible parameters as nuisance parameters. We formulate a new variation on such problems that arose in a study on the link between the number of cells in a charge-coupled detector image sensor that turned defective because of cosmic radiation and the storage conditions of such sensors. In this variation, the component densities are bivariate, there are two classes and only a subset of the mixture proportions is of relevance. We propose a nonparametric method to estimate the relevant subset of the proportions and apply our method to the data in our study.     

A note comparing component-slope,Scheffé and Cox parameterizations of the linear mixture experiment model

A mixture experiment involves combining two or more components in various proportions and collecting data on one or more responses. A linear mixture model may adequately represent the relationship between a response and mixture component proportions and be useful in screening the mixture components. The Scheffé and Cox parameterizations of the linear mixture model are commonly used for analyzing mixture experiment data. With the Scheffé parameterization, the fitted coefficient for a component is the predicted response at that pure component (i.e. single-component mixture). With the Cox parameterization, the fitted coefficient for a mixture component is the predicted difference in response at that pure component and at a pre-specified reference composition. This article presents a new component-slope parameterization, in which the fitted coefficient for a mixture component is the predicted slope of the linear response surface along the direction determined by that pure component and at a pre-specified reference composition. The component-slope, Scheffé, and Cox parameterizations of the linear mixture model are compared and their advantages and disadvantages are discussed.     

Ignorability, Sufficiency and Ancillarity

This paper explores the relationship between ignorability, sufficiency and ancillarity in the coarse data model of Heitjan and Rubin. Bayes or likelihood ignorability has a natural relationship to sufficiency, and frequentist ignorability an analogous relationship to ancillarity. Weaker conditions, termed observed likelihood sufficiency, observed specific sufficiency and observed ancillarity, expand the concepts to models where the coarsening mechanism is sometimes, but not always, ignorable.     

Modeling with Mixtures of Linear Regressions

Consider data (x ₁,y ₁),...,(x _n,y _n), where each x _i may be vector valued, and the distribution of y _i given x _i is a mixture of linear regressions. This provides a generalization of mixture models which do not include covariates in the mixture formulation. This mixture of linear regressions formulation has appeared in the computer science literature under the name Hierarchical Mixtures of Experts model.This model has been considered from both frequentist and Bayesian viewpoints. We focus on the Bayesian formulation. Previously, estimation of the mixture of linear regression model has been done through straightforward Gibbs sampling with latent variables. This paper contributes to this field in three major areas. First, we provide a theoretical underpinning to the Bayesian implementation by demonstrating consistency of the posterior distribution. This demonstration is done by extending results in Barron, Schervish and Wasserman (Annals of Statistics 27: 536–561, 1999) on bracketing entropy to the regression setting. Second, we demonstrate through examples that straightforward Gibbs sampling may fail to effectively explore the posterior distribution and provide alternative algorithms that are more accurate. Third, we demonstrate the usefulness of the mixture of linear regressions framework in Bayesian robust regression. The methods described in the paper are applied to two examples.