Parameter estimation for mixtures of skew Laplace normal distributions and application in mixture regression modeling

In this article, we propose mixtures of skew Laplace normal (SLN) distributions to model both skewness and heavy-tailedness in the neous data set as an alternative to mixtures of skew Student-t-normal (STN) distributions. We give the expectation–maximization (EM) algorithm to obtain the maximum likelihood (ML) estimators for the parameters of interest. We also analyze the mixture regression model based on the SLN distribution and provide the ML estimators of the parameters using the EM algorithm. The performance of the proposed mixture model is illustrated by a simulation study and two real data examples.     

Mini-batch learning of exponential family finite mixture models

Mini-batch algorithms have become increasingly popular due to the requirement for solving optimization problems, based on large-scale data sets. Using an existing online expectation–maximization (EM) algorithm framework, we demonstrate how mini-batch (MB) algorithms may be constructed, and propose a scheme for the stochastic stabilization of the constructed mini-batch algorithms. Theoretical results regarding the convergence of the mini-batch EM algorithms are presented. We then demonstrate how the mini-batch framework may be applied to conduct maximum likelihood (ML) estimation of mixtures of exponential family distributions, with emphasis on ML estimation for mixtures of normal distributions. Via a simulation study, we demonstrate that the mini-batch algorithm for mixtures of normal distributions can outperform the standard EM algorithm. Further evidence of the performance of the mini-batch framework is provided via an application to the famous MNIST data set.     

Likelihood-based inference for censored linear regression models with scale mixtures of skew-normal distributions

In many studies, the data collected are subject to some upper and lower detection limits. Hence, the responses are either left or right censored. A complication arises when these continuous measures present heavy tails and asymmetrical behavior; simultaneously. For such data structures, we propose a robust-censored linear model based on the scale mixtures of skew-normal (SMSN) distributions. The SMSN is an attractive class of asymmetrical heavy-tailed densities that includes the skew-normal, skew-t, skew-slash, skew-contaminated normal and the entire family of scale mixtures of normal (SMN) distributions as special cases. We propose a fast estimation procedure to obtain the maximum likelihood (ML) estimates of the parameters, using a stochastic approximation of the EM (SAEM) algorithm. This approach allows us to estimate the parameters of interest easily and quickly, obtaining as a byproducts the standard errors, predictions of unobservable values of the response and the log-likelihood function. The proposed methods are illustrated through real data applications and several simulation studies.     

Finite mixtures of multivariate Poisson distributions with application

In the present paper we examine finite mixtures of multivariate Poisson distributions as an alternative class of models for multivariate count data. The proposed models allow for both overdispersion in the marginal distributions and negative correlation, while they are computationally tractable using standard ideas from finite mixture modelling. An EM type algorithm for maximum likelihood (ML) estimation of the parameters is developed. The identifiability of this class of mixtures is proved. Properties of ML estimators are derived. A real data application concerning model based clustering for multivariate count data related to different types of crime is presented to illustrate the practical potential of the proposed class of models.     

A robust multivariate Birnbaum–Saunders distribution: EM estimation

We propose here a robust multivariate extension of the bivariate Birnbaum–Saunders (BS) distribution derived by Kundu et al. [Bivariate Birnbaum–Saunders distribution and associated inference. J Multivariate Anal. 2010;101:113–125], based on scale mixtures of normal (SMN) distributions that are used for modelling symmetric data. This resulting multivariate BS-type distribution is an absolutely continuous distribution whose marginal and conditional distributions are of BS-type distribution of Balakrishnan et al. [Estimation in the Birnbaum–Saunders distribution based on scalemixture of normals and the EM algorithm. Stat Oper Res Trans. 2009;33:171–192]. Due to the complexity of the likelihood function, parameter estimation by direct maximization is very difficult to achieve. For this reason, we exploit the nice hierarchical representation of the proposed distribution to propose a fast and accurate EM algorithm for computing the maximum likelihood (ML) estimates of the model parameters. We then evaluate the finite-sample performance of the developed EM algorithm and the asymptotic properties of the ML estimates through empirical experiments. Finally, we illustrate the obtained results with a real data and display the robustness feature of the estimation procedure developed here.     

The Two-Piece Normal-Laplace Distribution

This article considers the two-piece normal-Laplace (TPNL) distribution, a split skew distribution consisting of a normal part, and a Laplace part. The distribution is indexed by three parameters, representing location, scale, and shape. As illustrated with several examples, the TPNL family of distributions provides a useful alternative to other families of asymmetric distributions on the real line. However, because the likelihood function is not well behaved, standard theory of maximum-likelihood (ML) estimation does not apply to the TPNL family. In particular, the likelihood function can have multiple local maxima. We provide a procedure for computing ML estimators, and prove consistency and asymptotic normality of ML estimators, using non standard methods.     

Nonlinear semiparametric autoregressive model with finite mixtures of scale mixtures of skew normal innovations

We propose data generating structures which can be represented as the nonlinear autoregressive models with single and finite mixtures of scale mixtures of skew normal innovations. This class of models covers symmetric/asymmetric and light/heavy-tailed distributions, so provide a useful generalization of the symmetrical nonlinear autoregressive models. As semiparametric and nonparametric curve estimation are the approaches for exploring the structure of a nonlinear time series data set, in this article the semiparametric estimator for estimating the nonlinear function of the model is investigated based on the conditional least square method and nonparametric kernel approach. Also, an Expectation–Maximization-type algorithm to perform the maximum likelihood (ML) inference of unknown parameters of the model is proposed. Furthermore, some strong and weak consistency of the semiparametric estimator in this class of models are presented. Finally, to illustrate the usefulness of the proposed model, some simulation studies and an application to real data set are considered.     

On the robustness properties for maximum likelihood estimators of parameters in exponential power and generalized T distributions*

Abstract

Examining the robustness properties of maximum likelihood (ML) estimators of parameters in exponential power and generalized t distributions has been considered together. The well-known asymptotic properties of ML estimators of location, scale and added skewness parameters in these distributions are studied. The ML estimators for location, scale and scale variant (skewness) parameters are represented as an iterative reweighting algorithm (IRA) to compute the estimates of these parameters simultaneously. The artificial data are generated to examine performance of IRA for ML estimators of parameters simultaneously. We make a comparison between these two distributions to test the fitting performance on real data sets. The goodness of fit test and information criteria approve that robustness and fitting performance should be considered together as a key for modeling issue to have the best information from real data sets.     

Parametric estimation of mixtures of two uniform distributions

In this paper, we consider a mixture of two uniform distributions and derive L-moment estimators of its parameters. Three possible ways of mixing two uniforms, namely with neither overlap nor gap, with overlap, and with gap, are studied. The performance of these L-moment estimators in terms of bias and efficiency is compared to that obtained by means of the conventional method of moments (MM), modified maximum likelihood (MML) method and the usual maximum likelihood (ML) method. These intensive simulations reveal that MML estimators are the best in most of the cases, and the L-moment estimators are less subject to bias in estimation for some mixtures and more efficient in most of the cases than the conventional MM estimators. The L-moment estimators are, in some cases, more efficient than the ML and MML estimators.     

On the Estimation of Parameters of Kumaraswamy-G Distributions

Cordeiro and de Castro proposed a new family of generalized distributions based on the Kumaraswamy distribution (denoted as Kw-G). Nadarajah et al. showed that the density function of the new family of distributions can be expressed as a linear combination of the density of exponentiated family of distributions. They derived some properties of Kw-G distributions and discussed estimation of parameters using the maximum likelihood (ML) method. Cheng and Amin and Ranneby introduced a new method of estimating parameters based on Kullback–Leibler divergence (the maximum spacing (MSP) method). In this article, the estimates of parameters of Kw-G distributions are obtained using the MSP method. For some special Kw-G distributions, the new estimators are compared with ML estimators. It is shown by simulations and a real data application that MSP estimators have better properties than ML estimators.     

Finite mixtures of multivariate skew t-distributions: some recent and new results

Finite mixtures of multivariate skew t (MST) distributions have proven to be useful in modelling heterogeneous data with asymmetric and heavy tail behaviour. Recently, they have been exploited as an effective tool for modelling flow cytometric data. A number of algorithms for the computation of the maximum likelihood (ML) estimates for the model parameters of mixtures of MST distributions have been put forward in recent years. These implementations use various characterizations of the MST distribution, which are similar but not identical. While exact implementation of the expectation-maximization (EM) algorithm can be achieved for ‘restricted’ characterizations of the component skew t-distributions, Monte Carlo (MC) methods have been used to fit the ‘unrestricted’ models. In this paper, we review several recent fitting algorithms for finite mixtures of multivariate skew t-distributions, at the same time clarifying some of the connections between the various existing proposals. In particular, recent results have shown that the EM algorithm can be implemented exactly for faster computation of ML estimates for mixtures with unrestricted MST components. The gain in computational time is effected by noting that the semi-infinite integrals on the E-step of the EM algorithm can be put in the form of moments of the truncated multivariate non-central t-distribution, similar to the restricted case, which subsequently can be expressed in terms of the non-truncated form of the central t-distribution function for which fast algorithms are available. We present comparisons to illustrate the relative performance of the restricted and unrestricted models, and demonstrate the usefulness of the recently proposed methodology for the unrestricted MST mixture, by some applications to three real datasets.     

Diagnostics on nonlinear model with scale mixtures of skew-normal and first-order autoregressive errors

In this work, we develop some diagnostics for nonlinear regression model with scale mixtures of skew-normal (SMSN) and first-order autoregressive errors. The SMSN distribution class covers symmetric as well as asymmetric and heavy-tailed distributions, which offers a more flexible framework for modelling. Maximum-likelihood (ML) estimates are computed via an expectation–maximization-type algorithm. Local influence diagnostics and score test for the correlation are also derived. The performances of the ML estimates and the test statistic are investigated through Monte Carlo simulations. Finally, a real data set is used to illustrate our diagnostic methods.     

Measuring and estimating the interaction between exposures on a dichotomous outcome for observational studies

In observational studies for the interaction between exposures on a dichotomous outcome of a certain population, usually one parameter of a regression model is used to describe the interaction, leading to one measure of the interaction. In this article we use the conditional risk of an outcome given exposures and covariates to describe the interaction and obtain five different measures of the interaction, that is, difference between the marginal risk differences, ratio of the marginal risk ratios, ratio of the marginal odds ratios, ratio of the conditional risk ratios, and ratio of the conditional odds ratios. These measures reflect different aspects of the interaction. By using only one regression model for the conditional risk, we obtain the maximum-likelihood (ML)-based point and interval estimates of these measures, which are most efficient due to the nature of ML. We use the ML estimates of the model parameters to obtain the ML estimates of these measures. We use the approximate normal distribution of the ML estimates of the model parameters to obtain approximate non-normal distributions of the ML estimates of these measures and then confidence intervals of these measures. The method can be easily implemented and is presented via a medical example.     

Two-way ANOVA when the distribution of the error terms is skew t

In this article, we assume that the distribution of the error terms is skew t in two-way analysis of variance (ANOVA). Skew t distribution is very flexible for modeling the symmetric and the skew datasets, since it reduces to the well-known normal, skew normal, and Student's t distributions. We obtain the estimators of the model parameters by using the maximum likelihood (ML) and the modified maximum likelihood (MML) methodologies. We also propose new test statistics based on these estimators for testing the equality of the treatment and the block means and also the interaction effect. The efficiencies of the ML and the MML estimators and the power values of the test statistics based on them are compared with the corresponding normal theory results via Monte Carlo simulation study. Simulation results show that the proposed methodologies are more preferable. We also show that the test statistics based on the ML estimators are more powerful than the test statistics based on the MML estimators as expected. However, power values of the test statistics based on the MML estimators are very close to the corresponding test statistics based on the ML estimators. At the end of the study, a real life example is given to show the implementation of the proposed methodologies.     

Application of Dirichlet mixture of normals in growth curve models

In this paper, we present growth curve models with an auxiliary variable which contains an uncertain data distribution based on mixtures of standard components, such as normal distributions. The multimodality of the auxiliary random variable motivates and necessitates the use of mixtures of normal distributions in our model. We have observed that Dirichlet process priors, composed of discrete and continuous components, are appropriate in addressing the two problems of determining the number of components and estimating the parameters simultaneously and are especially useful in the aforementioned multimodal scenario. A model for the application of Dirichlet mixture of normals (DMN) in growth curve models under Bayesian formulation is presented and algorithms for computing the number of components, as well as estimating the parameters are also rendered. The simulation results show that our model gives improved goodness of fit statistics over models without DMN and the estimates for the number of components and for parameters are reasonably accurate.     

Homogeneity testing under finite location-scale mixtures

The testing problem for the order of finite mixture models has a long history and remains an active research topic. Since Ghosh & Sen (1985) revealed the hard-to-manage asymptotic properties of the likelihood ratio test, many successful alternative approaches have been developed. The most successful attempts include the modified likelihood ratio test and the EM-test, which lead to neat solutions for finite mixtures of univariate normal distributions, finite mixtures of single-parameter distributions, and several mixture-like models. The problem remains challenging, and there is still no generic solution for location-scale mixtures. In this article, we provide an EM-test solution for homogeneity for finite mixtures of location-scale family distributions. This EM-test has nonstandard limiting distributions, but we are able to find the critical values numerically. We use computer experiments to obtain appropriate values for the tuning parameters. A simulation study shows that the fine-tuned EM-test has close to nominal type I errors and very good power properties. Two application examples are included to demonstrate the performance of the EM-test.     

Mixture of Distributions in the Biparametric Exponential Family: A Bayesian Approach

In this article we propose mixture of distributions belonging to the biparametric exponential family, considering joint modeling of the mean and variance (or dispersion) parameters. As special cases we consider mixtures of normal and gamma distributions. A novel Bayesian methodology, using Markov Chain Monte Carlo (MCMC) methods, is proposed to obtain the posterior summaries of interest. We include simulations and real data examples to illustrate de performance of the proposal.     

Estimation of Mittag-Leffler Parameters

We propose a procedure for estimating the parameters of the Mittag-Leffler (ML) and the generalized Mittag-Leffler (GML) distributions. The algorithm is less restrictive, computationally simple, and necessary to make these models usable in practice. A comparison with the fractional moment estimator indicated favorable results for the proposed method.     

A use of mixtures of two normal distributions in a classification problem

An assumption made in the classification problem is that the distribution of the data being classified has the same parameters as the data used to obtain the discriminant functions. A method based on mixtures of two normal distributions is proposed as method of checking this assumption and modifying the discriminant functions accordingly. As a first step, the case considered in this paper, is that of a shift in the mean of one or two univariate normal distributions with all other parameters remaining fixed and known. Calculations based on the asymptotic the proposed method works well even for small shifts.     

Linear censored regression models with skew scale mixtures of normal distributions

A special source of difficulty in the statistical analysis is the possibility that some subjects may not have a complete observation of the response variable. Such incomplete observation of the response variable is called censoring. Censorship can occur for a variety of reasons, including limitations of measurement equipment, design of the experiment, and non-occurrence of the event of interest until the end of the study. In the presence of censoring, the dependence of the response variable on the explanatory variables can be explored through regression analysis. In this paper, we propose to examine the censorship problem in context of the class of asymmetric, i.e., we have proposed a linear regression model with censored responses based on skew scale mixtures of normal distributions. We develop a Monte Carlo EM (MCEM) algorithm to perform maximum likelihood inference of the parameters in the proposed linear censored regression models with skew scale mixtures of normal distributions. The MCEM algorithm has been discussed with an emphasis on the skew-normal, skew Student-t-normal, skew-slash and skew-contaminated normal distributions. To examine the performance of the proposed method, we present some simulation studies and analyze a real dataset.