Alternative modeling techniques for the quantal response data in mixture experiments

Mixture experiments are commonly encountered in many fields including chemical, pharmaceutical and consumer product industries. Due to their wide applications, mixture experiments, a special study of response surface methodology, have been given greater attention in both model building and determination of designs compared with other experimental studies. In this paper, some new approaches are suggested on model building and selection for the analysis of the data in mixture experiments by using a special generalized linear models, logistic regression model, proposed by Chen et al. [7]. Generally, the special mixture models, which do not have a constant term, are highly affected by collinearity in modeling the mixture experiments. For this reason, in order to alleviate the undesired effects of collinearity in the analysis of mixture experiments with logistic regression, a new mixture model is defined with an alternative ratio variable. The deviance analysis table is given for standard mixture polynomial models defined by transformations and special mixture models used as linear predictors. The effects of components on the response in the restricted experimental region are given by using an alternative representation of Cox's direction approach. In addition, odds ratio and the confidence intervals of odds ratio are identified according to the chosen reference and control groups. To compare the suggested models, some model selection criteria, graphical odds ratio and the confidence intervals of the odds ratio are used. The advantage of the suggested approaches is illustrated on tumor incidence data set.     

Unsupervised learning of regression mixture models with unknown number of components

ABSTRACT

We propose a new unsupervised learning algorithm to fit regression mixture models with unknown number of components. The developed approach consists in a penalized maximum likelihood estimation carried out by a robust expectation–maximization (EM)-like algorithm. We derive it for polynomial, spline, and B-spline regression mixtures. The proposed learning approach is unsupervised: (i) it simultaneously infers the model parameters and the optimal number of the regression mixture components from the data as the learning proceeds, rather than in a two-fold scheme as in standard model-based clustering using afterward model selection criteria, and (ii) it does not require accurate initialization unlike the standard EM for regression mixtures. The developed approach is applied to curve clustering problems. Numerical experiments on simulated and real data show that the proposed algorithm performs well and provides accurate clustering results, and confirm its benefit for practical applications.     

Optimal designs for experiments with mixtures: a survey

This is a survey article on known results about analytic solutions and numerical solutions of optimal designs for various regression models for experiments with mixtures. The regression models include polynomial models, models containing homogeneous functions, models containing inverse terms and ratios, log contrast models, models with quantitative variables, and mod els containing the amount of mixture, Optimality criteria considered include D-, A-, E-,φ_p- and I_λ-Optimalities. Uniform design and uniform optimal design for mixture components, and efficiencies of the {q,2} simplex-controid design are briefly discussed.     

A-optimal designs for optimum mixture in an additive quadratic mixture model

In the analysis of experiments with mixtures, quadratic models have been widely used. The optimum designs for the estimation of optimum mixing proportions in a quadratic mixture model have been studied by Pal and Mandal [Optimum designs for optimum mixtures. Statist Probab Lett. 2006;76:1369–1379] and Mandal et al. [Optimum mixture designs: a pseudo-Bayesian approach. J Ind Soc Agric Stat. 2008;62(2):174–182; Optimum mixture designs under constraints on mixing components. Statist Appl. 2008;6(1&2) (New Series): 189–205], using a pseudo-Bayesian approach. In this paper, a similar approach has been employed to obtain the A-optimal designs for the estimation of optimum proportions in an additive quadratic mixture model, proposed by Darroch and Waller [Additivity and interaction in three-component experiments with mixture. Biometrika. 1985;72:153–163], when the number of components is 3, 4 and 5. It has been shown that the vertices of the simplex are necessarily the support points of the optimum design, and the other support points include barycentres of depth at most 2.     

Finite mixture of regression models for a stratified sample

Despite the popularity and importance, there is limited work on modelling data which come from complex survey design using finite mixture models. In this work, we explored the use of finite mixture regression models when the samples were drawn using a complex survey design. In particular, we considered modelling data collected based on stratified sampling design. We developed a new design-based inference where we integrated sampling weights in the complete-data log-likelihood function. The expectation–maximisation algorithm was developed accordingly. A simulation study was conducted to compare the new methodology with the usual finite mixture of a regression model. The comparison was done using bias-variance components of mean square error. Additionally, a simulation study was conducted to assess the ability of the Bayesian information criterion to select the optimal number of components under the proposed modelling approach. The methodology was implemented on real data with good results.     

On Mixture Double Autoregressive Time Series Models

This article proposes a mixture double autoregressive model by introducing the flexibility of mixture models to the double autoregressive model, a novel conditional heteroscedastic model recently proposed in the literature. To make it more flexible, the mixing proportions are further assumed to be time varying, and probabilistic properties including strict stationarity and higher order moments are derived. Inference tools including the maximum likelihood estimation, an expectation–maximization (EM) algorithm for searching the estimator and an information criterion for model selection are carefully studied for the logistic mixture double autoregressive model, which has two components and is encountered more frequently in practice. Monte Carlo experiments give further support to the new models, and the analysis of an empirical example is also reported.     

MODELS AND DESIGNS FOR EXPERIMENTS WITH MIXTURES

Properties such as the tensile strength of an alloy of. different metals and the freezing point of a mixture of liquid chemicals, depend on the proportions (by weight or volume) of the components present and not on the total amount of the mixture. In choosing a model to relate such a property to the proportions of the various components of the mixture, there arise intriguing difficulties due to the fact that proportions sum to unity. It is demonstrated how to construct models which allow for the possibility of inactive components (components that do not affect the property at all) or components with additive effects. The design of experiments to fit such models to data is then discussed with a view to determining whether a given component is inactive or has an additive effect. The optimal allocation of observations to simplex-lattice designs is considered for one of these models. The construction of D -optimal designs for these models is an open problem.     

A graphical evaluation of logistic ridge estimator in mixture experiments

In comparison to other experimental studies, multicollinearity appears frequently in mixture experiments, a special study area of response surface methodology, due to the constraints on the components composing the mixture. In the analysis of mixture experiments by using a special generalized linear model, logistic regression model, multicollinearity causes precision problems in the maximum-likelihood logistic regression estimate. Therefore, effects due to multicollinearity can be reduced to a certain extent by using alternative approaches. One of these approaches is to use biased estimators for the estimation of the coefficients. In this paper, we suggest the use of logistic ridge regression (RR) estimator in the cases where there is multicollinearity during the analysis of mixture experiments using logistic regression. Also, for the selection of the biasing parameter, we use fraction of design space plots for evaluating the effect of the logistic RR estimator with respect to the scaled mean squared error of prediction. The suggested graphical approaches are illustrated on the tumor incidence data set.     

Mixture and mixture–process variable experiments for pharmaceutical applications

Many experiments in research and development in the pharmaceutical industry involve mixture components. These are experiments in which the experimental factors are the ingredients of a mixture and the response variable is a function of the relative proportion of each ingredient, not its absolute amount. Thus the mixture ingredients cannot be varied independently. A common variation of the mixture experiment occurs when there are also one or more process factors that can be varied independently of each other and of the mixture components, leading to a mixture–process variable experiment. We discuss the design and analysis of these types of experiments, using tablet formulation as an example. Our objective is to encourage greater utilization of these techniques in pharmaceutical research and development. Copyright © 2004 John Wiley & Sons Ltd.     

Symmetry in mixture experiments

Two types of symmetry can arise when the proportions of mixture components are constrained by upper and lower bounds. These two types of symmetry are shown to be useful for blocking first-order designs, as well as for finding the centroid of the experimental region. Orthogonal blocking of first-order mixture designs provides a method of including process variables in the mixture experiment, with the mixture terms orthogonal to the process factors. Symmetric regions are used to develop spherical and rotatable response surface designs for mixtures. The central composite design and designs based on the icosahedron and the dodecahedron are given for four-component mixtures. The uniform shell designs are three-level designs when applied to mixture experiments.     

A note comparing component-slope,Scheffé and Cox parameterizations of the linear mixture experiment model

A mixture experiment involves combining two or more components in various proportions and collecting data on one or more responses. A linear mixture model may adequately represent the relationship between a response and mixture component proportions and be useful in screening the mixture components. The Scheffé and Cox parameterizations of the linear mixture model are commonly used for analyzing mixture experiment data. With the Scheffé parameterization, the fitted coefficient for a component is the predicted response at that pure component (i.e. single-component mixture). With the Cox parameterization, the fitted coefficient for a mixture component is the predicted difference in response at that pure component and at a pre-specified reference composition. This article presents a new component-slope parameterization, in which the fitted coefficient for a mixture component is the predicted slope of the linear response surface along the direction determined by that pure component and at a pre-specified reference composition. The component-slope, Scheffé, and Cox parameterizations of the linear mixture model are compared and their advantages and disadvantages are discussed.     

DUAL‐OBJECTIVE OPTIMAL MIXTURE DESIGNS

Mixture experiments are widely used in many industries and particularly in the manufacture of consumer products. Almost all work to date assumes a single study objective, which is unrealistic. Researchers may want to estimate model parameters and make predictions or extrapolations at the same time. We discuss design issues for determining the optimal proportions of the mixture components when there are two or more objectives in the study and there is a large sample size. We present a general methodology for constructing two types of dual‐objective optimal design for mixture experiments and discuss the general applicability of the design strategy to more complicated types of mixture design problems, including mixture experiments.     

Design and analysis of mixture experiments with process variable

A mixture experiment is an experiment in which the response is assumed to depend on the relative proportions of the ingredients present in the mixture and not on the total amount of the mixture. In such experiment process, variables do not form any portion of the mixture but the levels changed could affect the blending properties of the ingredients. Sometimes, the mixture experiments are costly and the experiments are to be conducted in less number of runs. Here, a general method for construction of efficient mixture experiments in a minimum number of runs by the method for projection of efficient response surface design onto the constrained region is obtained. The efficient designs with a less number of runs have been constructed for 3rd, 4th, and 5th component of mixture experiments with one process variable.     

Mixture experiments with mixing errors

In mixture experiments the properties of mixtures are usually studied by mixing the amounts of the mixture components that are required to obtain the necessary proportions. This paper considers the impact of inaccuracies in discharging the required amounts of the mixture components on the statistical analysis of the data. It shows how the regression calibration approach can be used to minimize the resulting bias in the model and in the estimates of the model parameters, as well as to find correct estimates of the corresponding variances. Its application is made difficult by the complex structure of these errors. We also show how knowledge of the form of the model bias allows for choosing a manufacturing setting for a mixture product that is not biased and has smaller signal to noise ratio.     

Bayesian two-stage optimal design for mixture models

In this paper, a Bayesian two-stage D–D optimal design for mixture experimental models under model uncertainty is developed. A Bayesian D-optimality criterion is used in the first stage to minimize the determinant of the posterior variances of the parameters. The second stage design is then generated according to an optimalityprocedure that collaborates with the improved model from the first stage data. The results show that a Bayesian two-stage D–D-optimal design for mixture experiments under model uncertainty is more efficient than both the Bayesian one-stage D-optimal design and the non-Bayesian one-stage D-optimal design in most situations. Furthermore, simulations are used to obtain a reasonable ratio of the sample sizes between the two stages.     

A new variational Bayesian algorithm with application to human mobility pattern modeling

A new variational Bayesian (VB) algorithm, split and eliminate VB (SEVB), for modeling data via a Gaussian mixture model (GMM) is developed. This new algorithm makes use of component splitting in a way that is more appropriate for analyzing a large number of highly heterogeneous spiky spatial patterns with weak prior information than existing VB-based approaches. SEVB is a highly computationally efficient approach to Bayesian inference and like any VB-based algorithm it can perform model selection and parameter value estimation simultaneously. A significant feature of our algorithm is that the fitted number of components is not limited by the initial proposal giving increased modeling flexibility. We introduce two types of split operation in addition to proposing a new goodness-of-fit measure for evaluating mixture models. We evaluate their usefulness through empirical studies. In addition, we illustrate the utility of our new approach in an application on modeling human mobility patterns. This application involves large volumes of highly heterogeneous spiky data; it is difficult to model this type of data well using the standard VB approach as it is too restrictive and lacking in the flexibility required. Empirical results suggest that our algorithm has also improved upon the goodness-of-fit that would have been achieved using the standard VB method, and that it is also more robust to various initialization settings.     

Optimum Mixture Design for the Detection of Synergistic Effects

In mixture experiments, optimal designs for the estimation of parameters, both linear and non-linear, have been discussed by several authors. Optimal designs for the estimation of a subset of parameters have also been investigated. However, designs for testing the effects of certain factors and interactions have been studied only in the context of response surface models. In this article, we attempt to find the optimum design for testing the presence of synergistic effects in a mixture model. The classical F-test has been considered and the optimum design has been obtained so as to maximize the power of the test. It is observed that the barycenters are necessarily the support points of the trace-optimal design.     

Robust multivariate mixture regression models with incomplete data

Multivariate mixture regression models can be used to investigate the relationships between two or more response variables and a set of predictor variables by taking into consideration unobserved population heterogeneity. It is common to take multivariate normal distributions as mixing components, but this mixing model is sensitive to heavy-tailed errors and outliers. Although normal mixture models can approximate any distribution in principle, the number of components needed to account for heavy-tailed distributions can be very large. Mixture regression models based on the multivariate t distributions can be considered as a robust alternative approach. Missing data are inevitable in many situations and parameter estimates could be biased if the missing values are not handled properly. In this paper, we propose a multivariate t mixture regression model with missing information to model heterogeneity in regression function in the presence of outliers and missing values. Along with the robust parameter estimation, our proposed method can be used for (i) visualization of the partial correlation between response variables across latent classes and heterogeneous regressions, and (ii) outlier detection and robust clustering even under the presence of missing values. We also propose a multivariate t mixture regression model using MM-estimation with missing information that is robust to high-leverage outliers. The proposed methodologies are illustrated through simulation studies and real data analysis.     

Extensions of D-optimal minimal designs for symmetric mixture models

The purpose of mixture experiments is to explore the optimum blends of mixture components, which will provide the desirable response characteristics in finished products. D-optimal minimal designs have been considered for a variety of mixture models, including Scheffé's linear, quadratic, and cubic models. Usually, these D-optimal designs are minimally supported since they have just as many design points as the number of parameters. Thus, they lack the degrees of freedom to perform the lack-of-fit (LOF) tests. Also, the majority of the design points in D-optimal minimal designs are on the boundary: vertices, edges, or faces of the design simplex. In this article, extensions of the D-optimal minimal designs are developed for a general mixture model to allow additional interior points in the design space to enable prediction of the entire response surface. Also a new strategy for adding multiple interior points for symmetric mixture models is proposed. We compare the proposed designs with Cornell (1986 Cornell, J.A. (1986). A comparison between two ten-point designs for studying three-component mixture systems. J. Qual. Technol. 18(1):1–15.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]) two 10-point designs for the LOF test by simulations.     

Identifiable finite mixtures of location models for clustering mixed-mode data

For clustering mixed categorical and continuous data, Lawrence and Krzanowski (1996) proposed a finite mixture model in which component densities conform to the location model. In the graphical models literature the location model is known as the homogeneous Conditional Gaussian model. In this paper it is shown that their model is not identifiable without imposing additional restrictions. Specifically, for g groups and m locations, (g!)m–1 distinct sets of parameter values (not including permutations of the group mixing parameters) produce the same likelihood function. Excessive shrinkage of parameter estimates in a simulation experiment reported by Lawrence and Krzanowski (1996) is shown to be an artifact of the model's non-identifiability. Identifiable finite mixture models can be obtained by imposing restrictions on the conditional means of the continuous variables. These new identified models are assessed in simulation experiments. The conditional mean structure of the continuous variables in the restricted location mixture models is similar to that in the underlying variable mixture models proposed by Everitt (1988), but the restricted location mixture models are more computationally tractable.